Newest at the top
| 2026-06-30 19:46:14 +0000 | <monochrom> | Danse Macabre is a necromancy performance. >:) |
| 2026-06-30 19:45:39 +0000 | <int-e> | sounds more magical that way, like necromancy |
| 2026-06-30 19:45:23 +0000 | <int-e> | Hmm, "performancy" is cute. |
| 2026-06-30 19:45:03 +0000 | <int-e> | absentia: If you have no idea where the time is spent, profile first unless your code base *including* dependencies, except the ones whose performancy characteristics you understand perfectly is very small. |
| 2026-06-30 19:44:05 +0000 | merijn | (~merijn@host-cl.cgnat-g.v4.dfn.nl) merijn |
| 2026-06-30 19:44:02 +0000 | <tomsmeding> | int-e: both can be true at the same time |
| 2026-06-30 19:43:49 +0000 | <int-e> | Or few. |
| 2026-06-30 19:43:48 +0000 | <EvanR> | instead of guessing, blaming random things, and try to spread FUD |
| 2026-06-30 19:43:48 +0000 | <monochrom> | Yeah we need more "falsify your own hypotheses" attitudes. |
| 2026-06-30 19:43:37 +0000 | <int-e> | tomsmeding: Yes I have little constructive things to add ;) |
| 2026-06-30 19:43:29 +0000 | <absentia> | hnnngh |
| 2026-06-30 19:43:27 +0000 | <EvanR> | so you can get some more targeted claims more likely to be the problem |
| 2026-06-30 19:43:27 +0000 | <absentia> | yes it's always "profile first" isn't it |
| 2026-06-30 19:43:14 +0000 | <int-e> | Anyway. Profile, or read the library code trying to understand where the time might be spent. I'd think that profiling will be far less effort for starters. |
| 2026-06-30 19:43:09 +0000 | <EvanR> | if it wasn't obvious what would happen, I'd be temped to make minimal test cases to disprove all your claims, like it's impossible to quickly traverse 5000 items with a vector, etc |
| 2026-06-30 19:42:51 +0000 | <absentia> | i think the vector is strict |
| 2026-06-30 19:42:49 +0000 | <tomsmeding> | int-e: I've been saying this about 10 times throughout this conversation, in various phrasings |
| 2026-06-30 19:42:29 +0000 | <int-e> | tomsmeding: It's just, of all the bottlenecks I can imagine that isn't one of them. I suppose if it's a list that's populated lazily *that* can take a while too, and make it look as if iteration is slow... |
| 2026-06-30 19:42:00 +0000 | <absentia> | for this exact use case - ETL |
| 2026-06-30 19:41:56 +0000 | <absentia> | i've even done it before |
| 2026-06-30 19:41:54 +0000 | <absentia> | maybe i SHOULD just compile my own SQL |
| 2026-06-30 19:41:47 +0000 | <absentia> | good grief |
| 2026-06-30 19:41:42 +0000 | <tomsmeding> | not in traverse over a vector |
| 2026-06-30 19:41:38 +0000 | <tomsmeding> | if there is something massive, it's in hasql's internals here |
| 2026-06-30 19:41:32 +0000 | <absentia> | it's not, which is why this is so frustrating |
| 2026-06-30 19:41:27 +0000 | <absentia> | like max CPU |
| 2026-06-30 19:41:26 +0000 | <tomsmeding> | 5000 is _not_ massive |
| 2026-06-30 19:41:18 +0000 | <absentia> | yeah |
| 2026-06-30 19:41:15 +0000 | <absentia> | or intermediate result |
| 2026-06-30 19:41:14 +0000 | <tomsmeding> | and your haskell process is pegged all that time? |
| 2026-06-30 19:41:12 +0000 | <absentia> | so my presumption is the app layer is burning cycles building up some massive collection |
| 2026-06-30 19:41:10 +0000 | polykernel_ | polykernel |
| 2026-06-30 19:41:10 +0000 | polykernel | (~polykerne@user/polykernel) (Ping timeout: 276 seconds) |
| 2026-06-30 19:41:01 +0000 | <absentia> | i don't see any rows being inserted |
| 2026-06-30 19:40:57 +0000 | <absentia> | says everything is idle |
| 2026-06-30 19:40:57 +0000 | <tomsmeding> | if it's 30 minutes, that ought to be obvious |
| 2026-06-30 19:40:55 +0000 | <absentia> | i mean pg_stat_activity |
| 2026-06-30 19:40:51 +0000 | <tomsmeding> | like, open your favourite process monitor and look at what's using CPU |
| 2026-06-30 19:40:15 +0000 | <tomsmeding> | even apart from haskell profiling, have you checked whether that 30 minutes is spent in haskell or in postgres |
| 2026-06-30 19:40:07 +0000 | ski | (~ski@31-208-30-244.cust.bredband2.com) (Read error: Connection reset by peer) |
| 2026-06-30 19:40:02 +0000 | <absentia> | ik weird shit |
| 2026-06-30 19:40:02 +0000 | <monochrom> | What I do is I don't profile a completed project first. That's too late. If I have a hypothesis "I'm using State, I'm blaming State [denying that I'm also using a hundred other things so why am I singling out State]", I write 5 lines of code benchmarking State itself and nothing else. |
| 2026-06-30 19:39:59 +0000 | <tomsmeding> | ok |
| 2026-06-30 19:39:51 +0000 | <absentia> | in one network roundtrip |
| 2026-06-30 19:39:49 +0000 | <absentia> | it's a series of statements issued in postgresql pipeline mode |
| 2026-06-30 19:39:45 +0000 | <int-e> | tomsmeding: that's fine, it can still be expensive regardless ;-) |
| 2026-06-30 19:39:42 +0000 | <absentia> | it's neither exactly 5k SQL queries nor one query |
| 2026-06-30 19:39:22 +0000 | <tomsmeding> | but yes, I've been trying to say that a list of 5000 elements is not going to take 30 minutes; it's not even going to take 1ms |
| 2026-06-30 19:39:10 +0000 | <absentia> | but at that point why am i using a damn SQL library |
| 2026-06-30 19:39:00 +0000 | <absentia> | and just bang in VALUES (?, ?, ?), (?, ?, ?) |