Newest at the top
| 2025-11-13 14:25:30 +0100 | <kuribas`> | I hope we get rid of it as soon as possible, since it's maintained by a single guy, who is the only one to understand the whole system. |
| 2025-11-13 14:24:47 +0100 | Lycurgus | (~juan@user/Lycurgus) (Quit: alsoknownas.renjuan.org ( juan@acm.org )) |
| 2025-11-13 14:24:12 +0100 | <kuribas`> | And store in mysql on a triple store. |
| 2025-11-13 14:23:51 +0100 | <kuribas`> | __monty__: yes, everything except time series data. |
| 2025-11-13 14:23:23 +0100 | <__monty__> | The ORM loads all the data into memory? |
| 2025-11-13 14:22:54 +0100 | <kuribas`> | Since restarting the system takes an hour, to populate all the data in the ORM. |
| 2025-11-13 14:22:37 +0100 | <kuribas`> | Also the system is being kept up to date by keeping a running system, then patching it as we go. |
| 2025-11-13 14:22:10 +0100 | <kuribas`> | Also self-documenting values that are real-time inspectable in a UI (which hasn't been used for a decade now). |
| 2025-11-13 14:21:11 +0100 | <kuribas`> | __monty__: it's a whole lisp system with its own type system, an ORM based on a triple store, our own (graphql like) xml query engine, timeseries computations based on linked lists, etc... |
| 2025-11-13 14:19:57 +0100 | <__monty__> | That must be some implementation. |
| 2025-11-13 14:10:59 +0100 | <kuribas`> | Anyway, my current dict implementation is fast enough. For comparison, reading all the configuration files, and constructing this map in python takes less than a second, while in our legacy (lisp) system, it takes several minutes to load and store in the lisp ORM. |
| 2025-11-13 14:02:32 +0100 | CiaoSen | (~Jura@2a02:8071:64e1:da0:5a47:caff:fe78:33db) CiaoSen |
| 2025-11-13 14:01:35 +0100 | acarrico | (~acarrico@pppoe-209-99-223-51.greenmountainaccess.net) |
| 2025-11-13 13:59:47 +0100 | <[exa]> | yeah there's a very fast intersection algorithm for many skiplists |
| 2025-11-13 13:57:01 +0100 | <kuribas`> | [exa]: So you have three maps in parallel, then you intersect? |
| 2025-11-13 13:57:00 +0100 | <merijn> | something, something, lattice theory |
| 2025-11-13 13:55:38 +0100 | <[exa]> | (PS. skiplists are magical) |
| 2025-11-13 13:54:10 +0100 | <kuribas`> | hmm, interesting... |
| 2025-11-13 13:53:47 +0100 | <[exa]> | as in, in the 3-level map model, querying for (whatever, "bar", 3) requires traversing the whole first map; with inversion this is free |
| 2025-11-13 13:52:58 +0100 | <[exa]> | lucene version: make occurence inverted indexes for all key parts that you have, select Very Very Quickly by intersecting the skiplists. (That also gives you infinite wildcard queries for free.) |
| 2025-11-13 13:51:42 +0100 | <kuribas`> | True, putting * in the dictionary is less efficient. |
| 2025-11-13 13:51:07 +0100 | <[exa]> | kuribas`: can be highest; any predictable position is OK |
| 2025-11-13 13:50:51 +0100 | <kuribas`> | [exa]: why lowest? |
| 2025-11-13 13:49:51 +0100 | <kuribas`> | Maybe I can put an index at "a", and "b", since "c" (and d...) are sparse. |
| 2025-11-13 13:49:34 +0100 | <[exa]> | tree version: make * the lowest-ordering item, first select exactly, if that fails select minimum bound of ("foo","haha",whatever) to see if the wildcard is on the 3rd position (you can re-use the partial failed select!), then of ("foo", whatever, whatever) to see if it's on the 2nd position, ... |
| 2025-11-13 13:48:48 +0100 | CiaoSen | (~Jura@2a02:8071:64e1:da0:5a47:caff:fe78:33db) (Ping timeout: 256 seconds) |
| 2025-11-13 13:47:56 +0100 | <kuribas`> | yeah |
| 2025-11-13 13:47:17 +0100 | <[exa]> | ok I see, so whenever the `a` is not matched you try also the wildcard at that level, which also gives ordering to wildcards |
| 2025-11-13 13:46:24 +0100 | <kuribas`> | [exa]: no, something we use for configuration. |
| 2025-11-13 13:46:07 +0100 | <kuribas`> | [exa]: I have (HashMap (Wild a) (HashMap (Wild b) (HashMap (Wild c) val))) |
| 2025-11-13 13:45:57 +0100 | <lucabtz> | the wildcard is like a fallbacl |
| 2025-11-13 13:45:55 +0100 | <merijn> | kuribas`: Also, note that you can actually have SQLite run arbitrary custom functions mid query |
| 2025-11-13 13:45:51 +0100 | <[exa]> | kuribas`: is this RDF by the way? |
| 2025-11-13 13:45:44 +0100 | <lucabtz> | ah i see okay |
| 2025-11-13 13:45:34 +0100 | <merijn> | kuribas`: sorry, got distracted so missed half the context except the one question |
| 2025-11-13 13:45:19 +0100 | <kuribas`> | lucabtz: if you insert ((*, *, *) -> val), every lookup will succeed, but val only if everything else fails. |
| 2025-11-13 13:44:56 +0100 | merijn | (~merijn@77.242.116.146) merijn |
| 2025-11-13 13:44:28 +0100 | <[exa]> | (no other idea on how to map it to hashmaps) |
| 2025-11-13 13:44:13 +0100 | <lucabtz> | kuribas` so when setting say (*, *, *) you would modify the whole table? |
| 2025-11-13 13:43:59 +0100 | merijn | (~merijn@77.242.116.146) (Ping timeout: 256 seconds) |
| 2025-11-13 13:43:57 +0100 | <[exa]> | ah yes I meant that to get the result for ("foo","haha",3) you need to hash an dlookup the tuple as is, and then you need to hash and lookup (*, "haha", 3) and ("foo", *, 3), etc... |
| 2025-11-13 13:43:13 +0100 | <kuribas`> | each position can have a wildcard. |
| 2025-11-13 13:43:03 +0100 | <kuribas`> | [exa]: 3 |
| 2025-11-13 13:42:49 +0100 | <[exa]> | so your query algorithm basically checks 2 hash values for each position, right? |
| 2025-11-13 13:42:14 +0100 | <kuribas`> | but ordered by * last. So first the specific thing, only * if no specific thing exists. |
| 2025-11-13 13:41:39 +0100 | <kuribas`> | exactly |
| 2025-11-13 13:41:09 +0100 | <[exa]> | so these are keys that match all possible queries, as in, if you query for ("foo","haha",3) that thing with * is going to show up, right? |
| 2025-11-13 13:40:08 +0100 | <kuribas`> | [exa]: wildcard is only allowed when storing the keys, not when looking up. |
| 2025-11-13 13:39:22 +0100 | [exa] | confused |
| 2025-11-13 13:39:06 +0100 | <kuribas`> | [exa]: basically yes. Wildcard means match anything. |