| 2026-07-13 00:00:50 +0000 | vetkat5 | (~vetkat@user/vetkat) (Quit: So long, and thanks for all the fish) |
| 2026-07-13 00:01:33 +0000 | nisstyre | (wes@user/nisstyre) nisstyre |
| 2026-07-13 01:12:18 +0000 | xff0x | (~xff0x@2405:6580:b080:900:4828:98f5:f69e:d0ad) (Ping timeout: 248 seconds) |
| 2026-07-13 01:14:36 +0000 | spew | (~spew@user/spew) (Quit: nyaa~) |
| 2026-07-13 01:25:24 +0000 | tremon | (~tremon@83-80-159-219.cable.dynamic.v4.ziggo.nl) (Quit: getting boxed in) |
| 2026-07-13 01:29:13 +0000 | acidjnk_new | (~acidjnk@p200300d6e715132378c7491ce4365336.dip0.t-ipconnect.de) (Ping timeout: 248 seconds) |
| 2026-07-13 02:00:21 +0000 | weary-traveler | (~user@user/user363627) (Quit: Konversation terminated!) |
| 2026-07-13 02:00:37 +0000 | weary-traveler | (~user@user/user363627) user363627 |
| 2026-07-13 02:08:56 +0000 | xff0x | (~xff0x@fsb6a9491c.tkyc517.ap.nuro.jp) |
| 2026-07-13 02:13:16 +0000 | emilym | (~Thunderbi@user/emilym) emilym |
| 2026-07-13 02:16:07 +0000 | td_ | (~td@i53870918.versanet.de) (Ping timeout: 252 seconds) |
| 2026-07-13 02:17:45 +0000 | emilym | (~Thunderbi@user/emilym) (Ping timeout: 248 seconds) |
| 2026-07-13 02:22:56 +0000 | td_ | (~td@2001:9e8:19d0:7900:ee94:1637:4810:ed89) |
| 2026-07-13 02:27:11 +0000 | traxex | (traxex@user/traxex) (Ping timeout: 244 seconds) |
| 2026-07-13 03:01:13 +0000 | xkuru | (~xkuru@user/xkuru) xkuru |
| 2026-07-13 03:04:05 +0000 | weary-traveler | (~user@user/user363627) (Quit: Konversation terminated!) |
| 2026-07-13 03:04:21 +0000 | weary-traveler | (~user@user/user363627) user363627 |
| 2026-07-13 03:24:40 +0000 | bggd | (~bgg@user/bggd) bggd |
| 2026-07-13 03:28:39 +0000 | califax | (~califax@user/califx) (Remote host closed the connection) |
| 2026-07-13 03:30:58 +0000 | xkuru | (~xkuru@user/xkuru) (Quit: Unvirtualizing) |
| 2026-07-13 03:30:58 +0000 | califax | (~califax@user/califx) califx |
| 2026-07-13 03:46:22 +0000 | schuelermine | (~Thunderbi@user/schuelermine) (Ping timeout: 256 seconds) |
| 2026-07-13 04:48:43 +0000 | machinedgod | (~machinedg@d108-173-95-19.abhsia.telus.net) (Ping timeout: 276 seconds) |
| 2026-07-13 04:51:32 +0000 | czan | (~czan@user/mange) czan |
| 2026-07-13 05:05:30 +0000 | marinelli | (~weechat@gateway/tor-sasl/marinelli) (Quit: marinelli) |
| 2026-07-13 05:09:55 +0000 | echoreply | (~echoreply@45.32.163.16) (Ping timeout: 264 seconds) |
| 2026-07-13 05:18:52 +0000 | nyc | (~nyc@user/nyc) (Remote host closed the connection) |
| 2026-07-13 05:23:35 +0000 | echoreply | (~echoreply@2001:19f0:9002:1f3b:5400:ff:fe6f:8b8d) echoreply |
| 2026-07-13 05:29:44 +0000 | lisbeths | (uid135845@id-135845.lymington.irccloud.com) lisbeths |
| 2026-07-13 05:55:15 +0000 | haritz | (~hrtz@user/haritz) (Quit: ZNC 1.8.2+deb3.1+deb12u1 - https://znc.in) |
| 2026-07-13 05:57:54 +0000 | Sgeo | (~Sgeo@user/sgeo) (Read error: Connection reset by peer) |
| 2026-07-13 05:58:33 +0000 | traxex | (traxex@user/traxex) traxex |
| 2026-07-13 06:02:29 +0000 | mud | (~mud@user/kadoban) (Quit: quit) |
| 2026-07-13 06:03:04 +0000 | mud | (~mud@user/kadoban) kadoban |
| 2026-07-13 06:03:22 +0000 | mud | (~mud@user/kadoban) (Client Quit) |
| 2026-07-13 06:06:24 +0000 | mud | (~mud@user/kadoban) kadoban |
| 2026-07-13 06:43:35 +0000 | CiaoSen | (~Jura@2a02:3100:59f8:a800:4e50:ddff:fe9b:8922) CiaoSen |
| 2026-07-13 06:51:18 +0000 | sord937 | (~sord937@gateway/tor-sasl/sord937) sord937 |
| 2026-07-13 06:53:40 +0000 | jreicher | (~joelr@user/jreicher) (Quit: In transit) |
| 2026-07-13 07:05:25 +0000 | merijn | (~merijn@77.242.116.146) merijn |
| 2026-07-13 07:28:46 +0000 | tromp | (~textual@2001:1c00:340e:2700:70bb:dddb:6785:980d) |
| 2026-07-13 07:30:58 +0000 | <bwe> | jreicher: to see where the actual processing time is spent |
| 2026-07-13 07:31:27 +0000 | tzh | (~tzh@c-76-115-131-146.hsd1.or.comcast.net) (Quit: zzz) |
| 2026-07-13 07:35:28 +0000 | chele | (~chele@user/chele) chele |
| 2026-07-13 07:39:46 +0000 | <merijn> | bwe: A bit late to respond, but: tbh, I think getting parallelism/concurrency working in Haskell is comparatively easy to almost every other language, tbh. But yes, getting useful speedup is hard in any language and Haskell isn't magically different in that |
| 2026-07-13 07:40:52 +0000 | <merijn> | Especially when it comes to "large number of kinda small computations", that's a notoriously hard problem, tbh |
| 2026-07-13 08:06:10 +0000 | sam113102 | (~sam@modemcable200.189-202-24.mc.videotron.ca) sam113101 |
| 2026-07-13 08:06:29 +0000 | sam113101 | (~sam@modemcable200.189-202-24.mc.videotron.ca) (Ping timeout: 256 seconds) |
| 2026-07-13 08:06:31 +0000 | sam113102 | sam113101 |
| 2026-07-13 08:10:17 +0000 | Lord_of_Life | (~Lord@user/lord-of-life/x-2819915) (Excess Flood) |
| 2026-07-13 08:15:48 +0000 | Lord_of_Life | (~Lord@user/lord-of-life/x-2819915) Lord_of_Life |
| 2026-07-13 08:19:01 +0000 | <[exa]> | bwe: btw the easiest way to prevent laziness from causing problems is to print the results right from within the threads ;) |
| 2026-07-13 08:22:31 +0000 | tromp | (~textual@2001:1c00:340e:2700:70bb:dddb:6785:980d) (Quit: My iMac has gone to sleep. ZZZzzz…) |
| 2026-07-13 08:22:59 +0000 | ft | (~ft@p508dbe16.dip0.t-ipconnect.de) (Quit: leaving) |
| 2026-07-13 08:37:43 +0000 | jreicher | (~joelr@user/jreicher) jreicher |
| 2026-07-13 08:51:01 +0000 | __monty__ | (~toonn@user/toonn) toonn |
| 2026-07-13 08:53:02 +0000 | acidjnk_new | (~acidjnk@p200300d6e7151323a37ad98751598ed7.dip0.t-ipconnect.de) acidjnk |
| 2026-07-13 08:53:20 +0000 | fp | (~Thunderbi@130.233.70.229) fp |
| 2026-07-13 09:06:02 +0000 | <tomsmeding> | merijn: do you? While Haskell is nice in that it's high-level enough that you have easy concurrency tools available and a full scheduling RTS, you also have that in some other languages, and Haskell does make things harder with laziness (and the parallel GC) |
| 2026-07-13 09:07:06 +0000 | <tomsmeding> | STM is quite unique but I don't think you were talking about that |
| 2026-07-13 09:09:03 +0000 | <merijn> | tomsmeding: I mean those "some other languages" do not include "most mainstream ones" :p |
| 2026-07-13 09:09:09 +0000 | <tomsmeding> | Go? |
| 2026-07-13 09:09:21 +0000 | <merijn> | tomsmeding: The python, javascript, C, C++ stories are terrible |
| 2026-07-13 09:09:21 +0000 | <tomsmeding> | (I haven't really written any go, so perhaps I'm wrong in that) |
| 2026-07-13 09:09:29 +0000 | <tomsmeding> | yes |
| 2026-07-13 09:09:47 +0000 | <merijn> | Go is better, but still much worse than Haskell, in my experience (admittedly, like a decade ago) |
| 2026-07-13 09:09:55 +0000 | <tomsmeding> | c++ _threads_ are fine, although you don't have green threads, but the other concurrency tools are crappy |
| 2026-07-13 09:10:13 +0000 | <merijn> | tomsmeding: C++ is pretty decent, tbh. You just have to do **everything** yourself |
| 2026-07-13 09:10:19 +0000 | <merijn> | Same goes for C |
| 2026-07-13 09:10:49 +0000 | <merijn> | Hand rolling everything yourself using mutexes and CAS is a PITA compared to Haskell and the billion concurrent datatypes you have |
| 2026-07-13 09:10:56 +0000 | <tomsmeding> | right |
| 2026-07-13 09:11:53 +0000 | nschoe | (~nschoe@2a01:e0a:8e:a190:e469:96d4:b168:7f50) (Quit: ZNC 1.8.2 - https://znc.in) |
| 2026-07-13 09:11:54 +0000 | <merijn> | tomsmeding: So overall I stand by my "getting concurrency working in Haskell is easy compared to most languages" |
| 2026-07-13 09:12:09 +0000 | nschoe | (~nschoe@2a01:e0a:8e:a190:6ff0:ab82:de0a:883b) nschoe |
| 2026-07-13 09:12:09 +0000 | <merijn> | But getting it working != getting speedup ;) |
| 2026-07-13 09:12:35 +0000 | <tomsmeding> | sure, unless you mean that getting speedup is actually _harder_ than in other languages |
| 2026-07-13 09:12:44 +0000 | <merijn> | Then again, that holds in all the other languages too, so you have to do the same work to get speedup, but then you ALSO have to do the work to get it working in the first place |
| 2026-07-13 09:12:47 +0000 | <tomsmeding> | at which point I'm not sure if it's very useful |
| 2026-07-13 09:13:04 +0000 | <bwe> | merijn: I am right now in between both sides ;) |
| 2026-07-13 09:13:14 +0000 | <merijn> | I think getting speedup in Haskell is roughly as hard as in other languages |
| 2026-07-13 09:13:33 +0000 | <merijn> | Although, you did just remind me of an important question for bwe :p |
| 2026-07-13 09:13:33 +0000 | <tomsmeding> | like, it's not much harder for _me_ to get speedup in haskell than in other languages, because I have strong intuition for where laziness comes in and how to counteract that when necessary |
| 2026-07-13 09:13:44 +0000 | <tomsmeding> | getting that intuition is non-trivial |
| 2026-07-13 09:13:58 +0000 | <merijn> | That is, did anyone ask bwe whether he's actually forcing the result in each thread before returning a result? :p |
| 2026-07-13 09:14:34 +0000 | <merijn> | tomsmeding: Most of my concurrent code is heavily IO bound which makes forcing much simpler too |
| 2026-07-13 09:15:01 +0000 | <merijn> | tomsmeding: In the worst case, just deepseq before returning ;) |
| 2026-07-13 09:15:34 +0000 | <tomsmeding> | merijn: [exa] did, and bwe said he didn't |
| 2026-07-13 09:16:01 +0000 | <tomsmeding> | merijn: yes, but you have to put the deepseq in a Control.Exception.evaluate sometimes, otherwise it doesn't work |
| 2026-07-13 09:16:18 +0000 | <tomsmeding> | but in some situations you don't, which makes it more confusing because you can avoid learning it |
| 2026-07-13 09:16:30 +0000 | <tomsmeding> | I don't even fully understand when you need 'evaluate' |
| 2026-07-13 09:16:41 +0000 | <[exa]> | print is the best deepseq |
| 2026-07-13 09:16:43 +0000 | [exa] | hides |
| 2026-07-13 09:16:46 +0000 | <tomsmeding> | it is |
| 2026-07-13 09:16:54 +0000 | <tomsmeding> | unless you your value is big or you don't want it on stdout |
| 2026-07-13 09:17:30 +0000 | <tomsmeding> | my poor-man's deepseq for a while was ``length (show x) `seq` return ()``, but I'm 90% sure this didn't work for me at some point |
| 2026-07-13 09:17:50 +0000 | <tomsmeding> | (the seq-return part, not the length-show part, which is fine but crappy) |
| 2026-07-13 09:19:31 +0000 | CloneOfNone_ | (~CloneOfNo@user/CloneOfNone) (Ping timeout: 264 seconds) |
| 2026-07-13 09:19:38 +0000 | CloneOfNone | (~CloneOfNo@user/CloneOfNone) CloneOfNone |
| 2026-07-13 09:27:14 +0000 | marinelli | (~weechat@gateway/tor-sasl/marinelli) marinelli |
| 2026-07-13 09:29:56 +0000 | <bwe> | tomsmeding: I've setup two executables referencing the same Main, only difference is -threaded and -N for one. Verified that the threaded is using multiple cores by printing getNumCapabilities (threaded version prints 12, serial 1). |
| 2026-07-13 09:30:23 +0000 | <bwe> | tomsmeding: and here is the now compiling parseChunk function: https://paste.tomsmeding.com/GR3oWrvP |
| 2026-07-13 09:31:49 +0000 | <bwe> | tomsmeding: it fails on execution, however, since getChanContents blocks indefinitely. How do I get the contents of the Chan without blocking? |
| 2026-07-13 09:32:40 +0000 | <bwe> | Once that works, I can run serial and concurrent executables and see how long they take and how much they occupy the cores. |
| 2026-07-13 09:35:55 +0000 | <merijn> | tomsmeding: "evaluate is typically used to uncover any exceptions that a" |
| 2026-07-13 09:35:57 +0000 | <merijn> | lazy value may contain, |
| 2026-07-13 09:36:41 +0000 | <merijn> | bwe: getChanContents never ends, it produces a lazy list |
| 2026-07-13 09:36:47 +0000 | <merijn> | You probably don't want that |
| 2026-07-13 09:37:10 +0000 | <tomsmeding> | bwe: use replicateM (number of items that should be pushed) (readChan c) |
| 2026-07-13 09:37:26 +0000 | <tomsmeding> | bwe: I also see that your worker thread is not looping |
| 2026-07-13 09:37:38 +0000 | <merijn> | FYI, that code you're writing there is basically the rought prototype for the broadcast-chan function I linked you earlier :) |
| 2026-07-13 09:37:43 +0000 | <tomsmeding> | so if you push more than 12 work items on the channel, only the first 12 will be done |
| 2026-07-13 09:38:07 +0000 | <tomsmeding> | merijn: that may be my fault, I suggested to implement this basic thing because it's simple enough |
| 2026-07-13 09:38:13 +0000 | <merijn> | i.e. this one: https://hackage-content.haskell.org/package/broadcast-chan-0.3.0/docs/BroadcastChan.html#v:parFold… |
| 2026-07-13 09:39:05 +0000 | <bwe> | tomsmeding: it's fine, each chunk has length of nThreads |
| 2026-07-13 09:39:07 +0000 | <tomsmeding> | merijn: nice |
| 2026-07-13 09:40:06 +0000 | <merijn> | tomsmeding: That library was initially mostly about broadcast, but I was reimplementing that code bwe is doing so many times in different projects and the reality is *just* finnicky enough to be a PITA that I ended up just shoving it in there to make my life easier :p |
| 2026-07-13 09:40:19 +0000 | <tomsmeding> | right |
| 2026-07-13 09:40:40 +0000 | <merijn> | tomsmeding: That one is still kinda rough, there's also a conduit based one where you can just stream elements 1 at a time into an N parallel conduit, that's where the real love is |
| 2026-07-13 09:40:41 +0000 | <tomsmeding> | finicky because of the exceptions probably |
| 2026-07-13 09:41:02 +0000 | <merijn> | tomsmeding: Exceptions, ensuring backpressure, handling termination |
| 2026-07-13 09:41:13 +0000 | <merijn> | the `Chan` solution can never terminate, since Chan doesn't have a way to close |
| 2026-07-13 09:41:20 +0000 | <merijn> | So you never know when it's done |
| 2026-07-13 09:41:28 +0000 | <tomsmeding> | you know the number of items you pushed |
| 2026-07-13 09:41:37 +0000 | <tomsmeding> | but yeah backpressure is something I haven't thought about |
| 2026-07-13 09:41:42 +0000 | <merijn> | tomsmeding: IFF you communicate that separately from the Chan, sure |
| 2026-07-13 09:41:57 +0000 | <merijn> | tomsmeding: But then you need to write extra code and counters to check how much you've processed |
| 2026-07-13 09:43:39 +0000 | <merijn> | iirc (it's been a decade since I wrote it) that version keeps only 2*N elements "in flight", so lazy lists get forced as needed, instead of everything up front, potentially blowing up the heap with Chan contents |
| 2026-07-13 09:44:16 +0000 | <merijn> | tomsmeding: I just wanted something that handled all the termination, backpressure, exceptions, etc. in one go so I never had to think about it again :p |
| 2026-07-13 09:44:41 +0000 | <tomsmeding> | merijn: that backpressure functionality is neat! |
| 2026-07-13 09:44:58 +0000 | <tomsmeding> | yeah I've never had to care about exceptions or backpressure so far in the versions I've implemented I guess |
| 2026-07-13 09:46:10 +0000 | <bwe> | merijn: thanks for bringing that up again! I am yet to understand its utility, let me first get the manual approach working. |
| 2026-07-13 09:46:37 +0000 | <merijn> | tomsmeding: This is all stuff from my phd experiments. Where I was only allowed to reserve cluster nodes for 15 minutes max, and with individual experiments taking only a few second I was basically spinning up a node, pushing as many inputs into it until it got killed, but then I needed to 1) track elements lost in transit and 2) handle errors |
| 2026-07-13 09:47:20 +0000 | <merijn> | tomsmeding: because I was effectively reserving N nodes, pumping as many requests as possible in them until they got killed, then blocking until new reservations came online and continuing |
| 2026-07-13 09:48:04 +0000 | <tomsmeding> | that sounds like an adversarial environment lol |
| 2026-07-13 09:48:10 +0000 | <tomsmeding> | I've used this only in benign environments so far |
| 2026-07-13 09:48:22 +0000 | <merijn> | So for anything in the order of ~1s it should be a win in terms of performance. But it's been too long since I did benchmarking to know a "real" lowerbound on how short work can be, before the overhead becomes too much |
| 2026-07-13 09:48:55 +0000 | tromp | (~textual@2001:1c00:340e:2700:70bb:dddb:6785:980d) |
| 2026-07-13 09:49:02 +0000 | lisbeths | (uid135845@id-135845.lymington.irccloud.com) (Quit: Connection closed for inactivity) |
| 2026-07-13 09:49:13 +0000 | <merijn> | It's basically a regular Chan and some semaphores, so it should be pretty fast probably only in the range of a few hundred microsecond per element, but I haven't bothered testing :p |
| 2026-07-13 09:49:36 +0000 | <tomsmeding> | if your overhead is so much that it makes jobs <1s not worth it then you should revisit the implementation lol |
| 2026-07-13 09:49:41 +0000 | <tomsmeding> | a few hundred microseconds sounds more reasonable |
| 2026-07-13 09:49:50 +0000 | <bwe> | tomsmeding: https://paste.tomsmeding.com/U82xelTw <- runs now without failure; runtimes: serially: 6 sec; concurrently: 8 sec -- concurrently uses more cores but does not saturate them, serially shows one core spiking and then interestingly jumping to another core |
| 2026-07-13 09:50:15 +0000 | <merijn> | tomsmeding: It's just that 0.5-1s is the granularity I used it at, so that's all I've really tested :p |
| 2026-07-13 09:50:16 +0000 | <tomsmeding> | jumping to another core is just the OS scheduler occasionally moving a process to another core to spread heat production around the chip |
| 2026-07-13 09:50:58 +0000 | chromoblob | (~chromoblo@user/chromob1ot1c) (Ping timeout: 276 seconds) |
| 2026-07-13 09:51:01 +0000 | <merijn> | tomsmeding: I suspect you can probably even use it for jobs with tens/hundred of millisecond duration, maybe even shorter. But I've done nothing to ensure that nor tested it |
| 2026-07-13 09:51:14 +0000 | <merijn> | If someone wants to write some benchmarks to check that'd be neat |
| 2026-07-13 09:51:25 +0000 | fp | (~Thunderbi@130.233.70.229) (Remote host closed the connection) |
| 2026-07-13 09:51:42 +0000 | chromoblob | (~chromoblo@user/chromob1ot1c) chromoblob\0 |
| 2026-07-13 09:51:57 +0000 | <tomsmeding> | I suspect the same, you're too capable in this stuff to write code for this purpose where it wouldn't :p |
| 2026-07-13 09:51:59 +0000 | <bwe> | So, to recap, I've got the foundation right, using 12 cores, got the function to be executed by 1 vs. 12 cores. Now, it's time to investigate why there is no speedup. |
| 2026-07-13 09:51:59 +0000 | <merijn> | tomsmeding: My main usecase nowadays is farming out IO bound jobs (like fetching websites, encoding files, whatever) in parallel where the IO overhead dominates and the synchonisation cost is basically irrelevant |
| 2026-07-13 09:52:17 +0000 | <tomsmeding> | yeah that's strange |
| 2026-07-13 09:52:28 +0000 | <merijn> | bwe: With -threaded and -N12 you also **definitely** want -qg :) |
| 2026-07-13 09:52:35 +0000 | <tomsmeding> | have you tried +RTS -s? |
| 2026-07-13 09:52:36 +0000 | <[exa]> | bwe: just checking, do you have 1 selda backend connection per thread? |
| 2026-07-13 09:52:53 +0000 | <merijn> | oh, that's also a solid question :) |
| 2026-07-13 09:52:59 +0000 | <tomsmeding> | :D |
| 2026-07-13 09:53:27 +0000 | <[exa]> | bisect early™ |
| 2026-07-13 09:54:24 +0000 | <tomsmeding> | (it helps knowing what "Selda" is, I guess) |
| 2026-07-13 09:54:31 +0000 | <bwe> | [exa]: yes, I do. |
| 2026-07-13 09:55:04 +0000 | <tomsmeding> | is the work writing to or reading from the database? And is database access a significant part of what the threads should do? |
| 2026-07-13 09:55:20 +0000 | <tomsmeding> | because sqlite write access is sequential |
| 2026-07-13 09:55:32 +0000 | <bwe> | [exa]: the whole operation is running in a single withSQLite ... context. I already separated getting inputs from DB, parsing and writing back to DB. |
| 2026-07-13 09:56:11 +0000 | <bwe> | tomsmeding: that's why I want to run only the parsing concurrently. |
| 2026-07-13 09:56:26 +0000 | <tomsmeding> | oh wait, the threads are running in Reader Definitions, not in Selda |
| 2026-07-13 09:56:35 +0000 | <tomsmeding> | very good |
| 2026-07-13 09:57:17 +0000 | <tomsmeding> | ok, so the next things I'd want to know is +RTS -s, for sequential and for parallel, and failing useful information there, time measurements inside parseAndPrepToUpdate |
| 2026-07-13 09:57:47 +0000 | <tomsmeding> | oh, and also, bwe please replace `writeChan c r` with `writeChan c $! force r` |
| 2026-07-13 09:58:07 +0000 | <tomsmeding> | (just in case) |
| 2026-07-13 10:00:16 +0000 | tremon | (~tremon@83-80-159-219.cable.dynamic.v4.ziggo.nl) tremon |
| 2026-07-13 10:05:06 +0000 | CiaoSen | (~Jura@2a02:3100:59f8:a800:4e50:ddff:fe9b:8922) (Ping timeout: 248 seconds) |
| 2026-07-13 10:07:00 +0000 | <[exa]> | bwe: ok great (just the usual security questions :D ) |
| 2026-07-13 10:07:03 +0000 | luhann | (~luhann@user/luhann) (Quit: WeeChat 4.8.1) |
| 2026-07-13 10:07:16 +0000 | luhann | (~luhann@user/luhann) luhann |
| 2026-07-13 10:08:56 +0000 | <bwe> | tomsmeding: ghc-options: -threaded -rtsopts -with-rtsopts=-N -with-rtsopts=-s -- parallel variant |
| 2026-07-13 10:09:25 +0000 | <[exa]> | bwe: btw this is the latest version? https://paste.tomsmeding.com/U82xelTw |
| 2026-07-13 10:09:25 +0000 | <bwe> | tomsmeding: ghc-options: -rtsopts -with-rtsopts=-s -- serial variant |
| 2026-07-13 10:09:54 +0000 | <tomsmeding> | bwe: do also add -threaded to the serial variant to see if that inadvertently is making a difference (it ought not to, but who knows) |
| 2026-07-13 10:10:05 +0000 | <[exa]> | bwe: it looks like it _should_ be sequential because the readChan forces it to wait for the forkIO'd thread to finish, right? |
| 2026-07-13 10:10:24 +0000 | <[exa]> | oh no wrong sorry I'm stupid :D |
| 2026-07-13 10:10:32 +0000 | tomsmeding | was wondering what you meant |
| 2026-07-13 10:11:30 +0000 | <[exa]> | anyway yeah that exact code sample would still serialize because the `r` almost certainly remains a thunk |
| 2026-07-13 10:12:36 +0000 | <bwe> | [exa]: yes, it is |
| 2026-07-13 10:12:38 +0000 | <tomsmeding> | yes, hence my `writeChan c $! force r` suggestion |
| 2026-07-13 10:12:44 +0000 | <bwe> | here is the complete code: https://paste.tomsmeding.com/l9fvIZE4 |
| 2026-07-13 10:13:26 +0000 | <bwe> | tomsmeding: trying to start implementing the force brings me headache because of the return type inheritance (requiring me to add NFData instances all over places)! |
| 2026-07-13 10:13:50 +0000 | <tomsmeding> | bwe: well you can also try with just $! and without the force, if WHNF is enough |
| 2026-07-13 10:13:55 +0000 | <[exa]> | bwe: tbh just print it to /dev/null before the writeChan |
| 2026-07-13 10:14:01 +0000 | <tomsmeding> | but you're going to need to somehow ensure that the computation happens in the thread |
| 2026-07-13 10:14:15 +0000 | <bwe> | [exa]: see? I run everything in one withSQLite ... |
| 2026-07-13 10:14:23 +0000 | <[exa]> | bwe: yap that should be ok |
| 2026-07-13 10:14:31 +0000 | <tomsmeding> | without writing it to a _file_ (...) the other poor-man's version is ``length (show r) `seq` writeChan c r`` |
| 2026-07-13 10:14:53 +0000 | <[exa]> | bwe: just for the experiment, try doing `print . length $ show r` before the `writeChan c r` |
| 2026-07-13 10:15:01 +0000 | <tomsmeding> | or that |
| 2026-07-13 10:15:21 +0000 | <tomsmeding> | I checked and Chan is lazy in the values, so this is guaranteed to do all the work on the main thread and nothing useful in the worker threads |
| 2026-07-13 10:15:46 +0000 | <tomsmeding> | (it was laziness after all...) |
| 2026-07-13 10:15:47 +0000 | <[exa]> | bwe: also do you have a `htop` running or such, to see if the cores core well? |
| 2026-07-13 10:16:09 +0000 | <[exa]> | (htop -d1 helps) |
| 2026-07-13 10:16:29 +0000 | <tomsmeding> | [exa]: https://tirclogv.tomsmeding.com/log/haskell?eid=a6S4hHrBO#ev-a6S4hHrBO |
| 2026-07-13 10:16:52 +0000 | <bwe> | [exa]: zenith |
| 2026-07-13 10:17:29 +0000 | <bwe> | so, here is the output of -s (with print . length ...): https://paste.tomsmeding.com/VqIVSwJ8 |
| 2026-07-13 10:17:42 +0000 | <[exa]> | tomsmeding: "uses more cores but does not saturate" may read also as "jumps very actively between cores" |
| 2026-07-13 10:17:57 +0000 | <tomsmeding> | that would be a very stupid OS scheduler |
| 2026-07-13 10:19:57 +0000 | <[exa]> | nah that might be OK in the parallel case, the other thread usually waits on another core. (By RTS scheduler). |
| 2026-07-13 10:20:09 +0000 | <[exa]> | the "spikes then jumps to another core" sounds like OS scheduler. |
| 2026-07-13 10:20:31 +0000 | <[exa]> | bwe: okay at least we're back at 6 seconds I guess? :D |
| 2026-07-13 10:20:44 +0000 | <bwe> | tomsmeding: serial with -threaded: https://paste.tomsmeding.com/AmS4ckbi |
| 2026-07-13 10:21:12 +0000 | <bwe> | [exa]: it varies for both with each run. |
| 2026-07-13 10:21:16 +0000 | <tomsmeding> | 80% productivity is rather low |
| 2026-07-13 10:21:18 +0000 | <bwe> | (running a small sample) |
| 2026-07-13 10:21:25 +0000 | <tomsmeding> | 20% of your time (!) is spent in GC |
| 2026-07-13 10:21:25 +0000 | <bwe> | so, what to do next? |
| 2026-07-13 10:21:30 +0000 | <tomsmeding> | +RTS -qg |
| 2026-07-13 10:21:57 +0000 | <tomsmeding> | (also, hello fellow fish user) |
| 2026-07-13 10:22:06 +0000 | <[exa]> | bwe: can you try a slightly bigger sample to see if the amount of GC goes down or stays the same? Tbh this might not parallelize just because of the GC pressure. |
| 2026-07-13 10:28:31 +0000 | <[exa]> | (next step: just optimizing the software to 500x speed instead of parallelzition ;) ) |
| 2026-07-13 10:30:07 +0000 | danza | (~danza@user/danza) danza |
| 2026-07-13 10:32:10 +0000 | xff0x | (~xff0x@fsb6a9491c.tkyc517.ap.nuro.jp) (Ping timeout: 248 seconds) |
| 2026-07-13 10:34:36 +0000 | <bwe> | tomsmeding: -qg instead of -s or additionally? |
| 2026-07-13 10:35:24 +0000 | <tomsmeding> | -qg changes GC settings so that it runs on a single thread instead of on all capabilities (kernel threads) of the RTS; this shouldn't make a difference with -N1 but may well make a difference with -N |
| 2026-07-13 10:35:27 +0000 | <tomsmeding> | -s is just printing statistics |
| 2026-07-13 10:35:59 +0000 | <tomsmeding> | (perhaps counter-intuitively, making GC run on a single thread is often faster than parallel GC) |
| 2026-07-13 10:37:36 +0000 | <bwe> | -qg results are in: https://paste.tomsmeding.com/K0rssR46 |
| 2026-07-13 10:38:02 +0000 | <tomsmeding> | ok so that doesn't really help |
| 2026-07-13 10:38:50 +0000 | poscat | (~poscat@user/poscat) poscat |
| 2026-07-13 10:39:08 +0000 | fp | (~Thunderbi@130.233.70.229) fp |
| 2026-07-13 10:39:12 +0000 | <[exa]> | bwe: would you mind a profile build, just to see where it spends time? (not sure if you profiled already) |
| 2026-07-13 10:39:17 +0000 | <tomsmeding> | bwe: can you use the 'clock' package and print the diffTimeSpec of getTime at the start and at the end of the worker thread body? |
| 2026-07-13 10:39:40 +0000 | <tomsmeding> | profiling is also possible but more finicky and possibly harder to read |
| 2026-07-13 10:40:13 +0000 | <tomsmeding> | for getTime, use either Realtime or Monotonic, it shouldn't really matter much here (but use the same on both calls) |
| 2026-07-13 10:40:16 +0000 | <[exa]> | bwe: in short run cabal with --enable-profiling and then run with +RTS +p |
| 2026-07-13 10:40:31 +0000 | poscat0x04 | (~poscat@user/poscat) (Ping timeout: 264 seconds) |
| 2026-07-13 10:40:53 +0000 | <[exa]> | (it might take some time to recompile stuff with profiling support but hopefully not too much) |
| 2026-07-13 10:41:05 +0000 | sim590 | (~sim590@209-15-184-6.resi.cgocable.ca) sim590 |
| 2026-07-13 10:42:08 +0000 | <bwe> | [exa]: no change in running a bigger sample (≈ 100 secs for both variants) |
| 2026-07-13 10:42:09 +0000 | CiaoSen | (~Jura@2a02:3100:59f8:a800:4e50:ddff:fe9b:8922) CiaoSen |
| 2026-07-13 10:42:33 +0000 | <[exa]> | bwe: did the percent at MUT go up or down somewhat significantly? you had ~80% there before |
| 2026-07-13 10:42:45 +0000 | <bwe> | tomsmeding: will do that later and report back |
| 2026-07-13 10:42:57 +0000 | <[exa]> | bwe: cf line 13 here https://paste.tomsmeding.com/AmS4ckbi |
| 2026-07-13 10:43:50 +0000 | <[exa]> | s/MUT/productivity/ oh names |
| 2026-07-13 10:46:08 +0000 | tromp | (~textual@2001:1c00:340e:2700:70bb:dddb:6785:980d) (Quit: My iMac has gone to sleep. ZZZzzz…) |
| 2026-07-13 10:46:38 +0000 | lisbeths | (uid135845@id-135845.lymington.irccloud.com) lisbeths |
| 2026-07-13 10:59:13 +0000 | tromp | (~textual@2001:1c00:340e:2700:70bb:dddb:6785:980d) |
| 2026-07-13 10:59:31 +0000 | <merijn> | tomsmeding: SQLite write access is definitely not sequential |
| 2026-07-13 11:00:22 +0000 | <merijn> | bwe: Don't multiple --writh-rtsopts clobber each other? |
| 2026-07-13 11:09:30 +0000 | <merijn> | I would think you'd need `--with-rtsopts="-N -qg -s"` |
| 2026-07-13 11:10:28 +0000 | <merijn> | tomsmeding, [exa], bwe: Also, without -qg the GC will run parallel on every capability, leading to it looking like every thread is active while it's just wasting cycles on GC :p |
| 2026-07-13 11:11:09 +0000 | <merijn> | 78% productivity isn't great, but it's also not atrocious, tbh |
| 2026-07-13 11:13:34 +0000 | <merijn> | I'm suspicious whether 1) Selda configures SQLite for concurrent writes and 2) whether it uses pooling so every worker has a pool. I'm 90% certain not |
| 2026-07-13 11:13:45 +0000 | picnoir | (~picnoir@about/aquilenet/vodoo/NinjaTrappeur) (Ping timeout: 248 seconds) |
| 2026-07-13 11:13:48 +0000 | <merijn> | So that might end up forcing sequential behaviour regardless |
| 2026-07-13 11:23:07 +0000 | xff0x | (~xff0x@ai070051.d.east.v6connect.net) |
| 2026-07-13 11:26:07 +0000 | picnoir | (~picnoir@about/aquilenet/vodoo/NinjaTrappeur) NinjaTrappeur |
| 2026-07-13 11:28:18 +0000 | Lord_of_Life | (~Lord@user/lord-of-life/x-2819915) (Excess Flood) |
| 2026-07-13 11:28:26 +0000 | <[exa]> | but it gets all written all the way in the end right? |
| 2026-07-13 11:30:29 +0000 | Lord_of_Life | (~Lord@user/lord-of-life/x-2819915) Lord_of_Life |
| 2026-07-13 11:31:49 +0000 | rabbull7217 | (~rabbull@xdsl-31-164-93-219.adslplus.ch) (Quit: Ping timeout (120 seconds)) |
| 2026-07-13 11:32:00 +0000 | rabbull7217 | (~rabbull@xdsl-31-164-93-219.adslplus.ch) |
| 2026-07-13 11:34:18 +0000 | <merijn> | [exa]: Yeah, but if you use a single SQL connection you might hit locks in the connection on parallel accesses |
| 2026-07-13 11:34:30 +0000 | <merijn> | i.e. only one transaction running at a time for the connection |
| 2026-07-13 11:34:55 +0000 | Lord_of_Life | (~Lord@user/lord-of-life/x-2819915) (Ping timeout: 252 seconds) |
| 2026-07-13 11:35:03 +0000 | <[exa]> | it doesn't seem to me that the SQL would be a part of the bottleneck, the actual computation looks pure there |
| 2026-07-13 11:35:28 +0000 | <[exa]> | (provided ofc. the computation is actually what takes the time there) |
| 2026-07-13 11:39:29 +0000 | picnoir | (~picnoir@about/aquilenet/vodoo/NinjaTrappeur) (Quit: WeeChat 4.9.1) |
| 2026-07-13 11:39:50 +0000 | Lord_of_Life | (~Lord@user/lord-of-life/x-2819915) Lord_of_Life |
| 2026-07-13 11:41:45 +0000 | haritz | (~hrtz@2a01:4b00:bc2e:7000:d5af:a266:ca31:5ef8) |
| 2026-07-13 11:41:45 +0000 | haritz | (~hrtz@2a01:4b00:bc2e:7000:d5af:a266:ca31:5ef8) (Changing host) |
| 2026-07-13 11:41:45 +0000 | haritz | (~hrtz@user/haritz) haritz |
| 2026-07-13 11:43:07 +0000 | picnoir | (~picnoir@about/aquilenet/vodoo/NinjaTrappeur) NinjaTrappeur |
| 2026-07-13 11:44:41 +0000 | Lord_of_Life | (~Lord@user/lord-of-life/x-2819915) (Ping timeout: 265 seconds) |
| 2026-07-13 11:44:53 +0000 | danz37027 | (~danza@user/danza) danza |
| 2026-07-13 11:45:49 +0000 | danza | (~danza@user/danza) (Read error: Connection reset by peer) |
| 2026-07-13 11:46:49 +0000 | <bwe> | merijn: I stream the entries from DB within single connection to SQLite (as I understand it) then make up chunks equivalent to no. of cores then fork core no. of threads, write it to Chan, collect the chan (per chunk), write it to DB (for each chunk). |
| 2026-07-13 11:47:55 +0000 | tromp | (~textual@2001:1c00:340e:2700:70bb:dddb:6785:980d) (Quit: My iMac has gone to sleep. ZZZzzz…) |
| 2026-07-13 11:48:48 +0000 | bitdex | (~bitdex@gateway/tor-sasl/bitdex) (Quit: = "") |
| 2026-07-13 11:49:48 +0000 | <merijn> | Incidentally, this is pretty much my use-case for broadcast-chan, except I used persistent, so sadly you can't directly crib my homework (and I don't know I'd recommend switching to persistent, I mostly used it due to sunk cast fallacy :p) |
| 2026-07-13 11:50:25 +0000 | czan | (~czan@user/mange) (Quit: Zzz...) |
| 2026-07-13 11:50:47 +0000 | <bwe> | merijn: the problem is that package.yaml doesn't accept this multiple syntax of rtsopts :( |
| 2026-07-13 11:51:47 +0000 | <merijn> | Welp, that sounds like a problem with package.yaml :p |
| 2026-07-13 11:52:01 +0000 | <merijn> | Incidentally, you can just specify RTS opts at runtime when compiling with -rtsopts |
| 2026-07-13 11:52:17 +0000 | <merijn> | By running the resulting executable with +RTS <options here> |
| 2026-07-13 11:52:31 +0000 | <merijn> | oh |
| 2026-07-13 11:53:40 +0000 | <merijn> | There was a flag for dumping out the active RTS opts, maybe try that and see if the multiple --with-rtsopts is actually working? |
| 2026-07-13 11:54:47 +0000 | <merijn> | That said, I would just ditch package.yaml, it's stupid IMO |
| 2026-07-13 11:56:04 +0000 | <tomsmeding> | bwe: please pass the options directly using +RTS, at least that certainly works |
| 2026-07-13 11:56:08 +0000 | <int-e> | ghc +RTS --info says ",("Flag -with-rtsopts", "-K512M -H -I5 -T")" among other things |
| 2026-07-13 11:56:55 +0000 | <int-e> | (that's an RTS feature, distinct from ghc's own --info) |
| 2026-07-13 11:59:48 +0000 | <bwe> | new results with directly passing options: https://paste.tomsmeding.com/DiDxHua8 |
| 2026-07-13 12:01:44 +0000 | caskd | (~caskd@user/caskd) (Remote host closed the connection) |
| 2026-07-13 12:02:10 +0000 | caskd | (~caskd@user/caskd) caskd |
| 2026-07-13 12:03:06 +0000 | Guest6 | (~Guest6@147.161.130.251) |
| 2026-07-13 12:05:12 +0000 | <bwe> | merijn: why is package.yaml stupid? |
| 2026-07-13 12:06:08 +0000 | <tomsmeding> | bwe: your "concurrently" output shows "using -N1" which means that the -N is not getting passed correctly it seems |
| 2026-07-13 12:06:25 +0000 | <tomsmeding> | e.g. if I type `ghc +RTS -s -N2` I do get "using -N2" |
| 2026-07-13 12:17:22 +0000 | weary-traveler | (~user@user/user363627) (Remote host closed the connection) |
| 2026-07-13 12:20:36 +0000 | <bwe> | with diffTimeSpec but still unclear whether -N is getting passed through: https://paste.tomsmeding.com/gKGL0IFW |
| 2026-07-13 12:21:11 +0000 | <bwe> | ah, it shows using -N12 now, so it should be correct now! |
| 2026-07-13 12:22:28 +0000 | <bwe> | so, saving each chunk is taking quite some time; also for some reason the parsing time varies quite significantly. |
| 2026-07-13 12:23:07 +0000 | <bwe> | still, now the -N1 takes ≈ 9 secs while the -N1 takes ≈ 6 secs. |
| 2026-07-13 12:23:34 +0000 | tromp | (~textual@2001:1c00:340e:2700:70bb:dddb:6785:980d) |
| 2026-07-13 12:24:27 +0000 | Alex_delenda_est | (~al_test@178.34.160.250) (Ping timeout: 244 seconds) |
| 2026-07-13 12:24:27 +0000 | AlexZenon | (~alzenon@178.34.160.250) (Ping timeout: 244 seconds) |
| 2026-07-13 12:24:40 +0000 | Lord_of_Life | (~Lord@user/lord-of-life/x-2819915) Lord_of_Life |
| 2026-07-13 12:24:50 +0000 | <tomsmeding> | which of those should be -N12? |
| 2026-07-13 12:25:13 +0000 | <bwe> | the 9 sec, sorry |
| 2026-07-13 12:25:33 +0000 | <merijn> | bwe: To begin with YAML is kind of a terrible format. package.yaml is just a rather pointless wrapper/frontend for generating a cabal file from a yaml file. But since it's not a first-class format used/understood by haskell tooling you end up with problems like yours of valid configurations like not working |
| 2026-07-13 12:25:43 +0000 | AlexNoo | (~AlexNoo@178.34.160.250) (Ping timeout: 256 seconds) |
| 2026-07-13 12:26:19 +0000 | <merijn> | bwe: The only real argument "for" package.yaml appears to be support for wildcard source inclusions which (imo) is an active "anti-feature", which is why cabal doesn't support it |
| 2026-07-13 12:26:21 +0000 | califax | (~califax@user/califx) (Remote host closed the connection) |
| 2026-07-13 12:26:31 +0000 | xdej | (~xdej@quatramaran.salle-s.org) xdej |
| 2026-07-13 12:26:40 +0000 | califax | (~califax@user/califx) califx |
| 2026-07-13 12:27:20 +0000 | <bwe> | merijn: admittedly, I have to review the latest cabal file standard, I remember there are many versions which held me off to consider using it (I might be wrong) |
| 2026-07-13 12:28:39 +0000 | <fp> | Is YAML bad just because of the silly reference stuff? Would a subset without that be good? |
| 2026-07-13 12:28:57 +0000 | <fp> | I guess the way it's lexed is also dumb |
| 2026-07-13 12:29:02 +0000 | <bwe> | tomsmeding: here is how I measure diffTimeSpec: https://paste.tomsmeding.com/xgKnHSki |
| 2026-07-13 12:29:20 +0000 | <merijn> | fp: Yaml is bad for lots and lots of reasons. It is stupidly complex with many special cases. It's impossible to tell whether a yaml file has been truncated due to (for example) network error |
| 2026-07-13 12:29:39 +0000 | Guest6 | (~Guest6@147.161.130.251) (Quit: Client closed) |
| 2026-07-13 12:30:00 +0000 | Lord_of_Life | (~Lord@user/lord-of-life/x-2819915) (Excess Flood) |
| 2026-07-13 12:30:25 +0000 | <fp> | Ah interesting |
| 2026-07-13 12:30:37 +0000 | <merijn> | fp: By supporting a bazillion different options to write things it makes it actively difficult as human to understand what any given file means. It does annoying unexpected things like "magically turn ISO dates into pretty printed data strings", has 36 different ways to write booleans. And essentially 0 of the parser implementations interpret the format the same |
| 2026-07-13 12:31:33 +0000 | <fp> | I have heard that you have to quote pretty aggressively to avoid some of the BS |
| 2026-07-13 12:31:56 +0000 | <tomsmeding> | bwe: I meant timing each individual worker thread, like this: https://paste.tomsmeding.com/O7kyq4co |
| 2026-07-13 12:32:02 +0000 | <sm> | bwe, "package.yaml doesn't accept this multiple syntax of rtsopts" - are you sure ? |
| 2026-07-13 12:32:17 +0000 | <merijn> | fp: Right, but I wouldn't consider "activelly defensively writing a file format to not get boobytrapped" a rather significant anti-feature ;) |
| 2026-07-13 12:32:31 +0000 | <fp> | Oh sure |
| 2026-07-13 12:33:02 +0000 | <fp> | I just use it instead of JSON because it supports integers |
| 2026-07-13 12:33:30 +0000 | <fp> | Really I should be writing everything as sexps |
| 2026-07-13 12:34:00 +0000 | Lord_of_Life | (~Lord@user/lord-of-life/x-2819915) Lord_of_Life |
| 2026-07-13 12:37:01 +0000 | <bwe> | tomsmeding: https://paste.tomsmeding.com/AqajXLxH |
| 2026-07-13 12:39:10 +0000 | <tomsmeding> | that parallel productivity is pretty good suddenly |
| 2026-07-13 12:39:38 +0000 | <bwe> | still parallel takes longer in total |
| 2026-07-13 12:41:28 +0000 | <tomsmeding> | mean task processing time in the serial version is larger |
| 2026-07-13 12:41:44 +0000 | <tomsmeding> | 0.41sec instead of 0.31sec for serial |
| 2026-07-13 12:41:50 +0000 | <tomsmeding> | uh |
| 2026-07-13 12:42:01 +0000 | <tomsmeding> | serial 0.41sec, parallel 0.31sec |
| 2026-07-13 12:42:32 +0000 | <tomsmeding> | interesting |
| 2026-07-13 12:43:11 +0000 | <tomsmeding> | bwe: "saving" is the step after this? |
| 2026-07-13 12:43:32 +0000 | <merijn> | 98% producitivity looking pretty nice |
| 2026-07-13 12:43:53 +0000 | <tomsmeding> | yes but it's surprising that productivity goes up that much if the time spent processing an item goes _up_ on average, and you add more threads |
| 2026-07-13 12:44:29 +0000 | <merijn> | That seems pretty obvious? |
| 2026-07-13 12:44:31 +0000 | <merijn> | Also |
| 2026-07-13 12:44:44 +0000 | <merijn> | Note that the printing in every thread is introducing serialisation via stdout |
| 2026-07-13 12:44:58 +0000 | <merijn> | Rather ruining this entire benchmark entirely |
| 2026-07-13 12:45:43 +0000 | <merijn> | Key take away from my phd: Benchmarking is HARD. Benchmarking incredibly small amounts of work is orders of magnitude harder |
| 2026-07-13 12:47:23 +0000 | rekahsoft | (~rekahsoft@70.51.99.119) rekahsoft |
| 2026-07-13 12:48:39 +0000 | <tomsmeding> | merijn: the printed runtimes are on the order of 0.3-0.4 seconds; printing does not meaningfully impact that |
| 2026-07-13 12:48:58 +0000 | <tomsmeding> | that's the weird part about all of this, all these effects make perfect sense if this was about 0.3-0.4 ms |
| 2026-07-13 12:49:02 +0000 | <tomsmeding> | but it's _seconds_ |
| 2026-07-13 12:50:09 +0000 | <tomsmeding> | (or, rather, if it was about 0.3 ms you'd expect things to not make sense with this setup, so everything would go) |
| 2026-07-13 12:51:03 +0000 | <tomsmeding> | though I agree with your "benchmarking is hard" :p |
| 2026-07-13 12:51:12 +0000 | kilolympus0 | (~kilolympu@vmi1102682.contaboserver.net) (Quit: Ping timeout (120 seconds)) |
| 2026-07-13 12:51:32 +0000 | kilolympus0 | (~kilolympu@vmi1102682.contaboserver.net) kilolympus |
| 2026-07-13 12:51:38 +0000 | <tomsmeding> | I'm dealing with stuff where the measured time of some routines depends fairly significantly on ASLR because things either overlap or not in CPU caches |
| 2026-07-13 12:51:56 +0000 | <tomsmeding> | fortunately those are the stupid routines and they're just there for being slow and a baseline |
| 2026-07-13 12:55:01 +0000 | <merijn> | tomsmeding: The problem is that your worker threads are blocking on each other in the event loop to print |
| 2026-07-13 12:55:25 +0000 | <tomsmeding> | merijn: yes, for like a millisecond or 2 |
| 2026-07-13 12:55:36 +0000 | <tomsmeding> | that does not add three seconds to the total runtime |
| 2026-07-13 12:55:50 +0000 | <tomsmeding> | (2ms is already generous) |
| 2026-07-13 12:56:17 +0000 | <bwe> | tomsmeding: so, what's the main takeaway and course of action? |
| 2026-07-13 12:56:48 +0000 | <merijn> | tomsmeding: I'm not sure what the queueing behaviour is for writing to the same file descriptor in multiple threads. It's definitely something I would avoid if I wanted remotely accurate numbers |
| 2026-07-13 12:56:53 +0000 | sord937 | (~sord937@gateway/tor-sasl/sord937) (Quit: sord937) |
| 2026-07-13 12:58:24 +0000 | chromoblob | (~chromoblo@user/chromob1ot1c) (Read error: Connection reset by peer) |
| 2026-07-13 12:58:25 +0000 | <tomsmeding> | merijn: https://paste.tomsmeding.com/Q54duCms |
| 2026-07-13 12:58:36 +0000 | <tomsmeding> | this isn't being an issue |
| 2026-07-13 12:58:57 +0000 | kilolympus0 | (~kilolympu@vmi1102682.contaboserver.net) (Quit: The Lounge - https://thelounge.chat) |
| 2026-07-13 12:59:10 +0000 | chromoblob | (~chromoblo@user/chromob1ot1c) chromoblob\0 |
| 2026-07-13 13:00:43 +0000 | <tomsmeding> | bwe: I'm out of ideas for debugging this remotely, perhaps other people are not. I'd want to tinker with this myself at this point |
| 2026-07-13 13:01:02 +0000 | AlexNoo | (~AlexNoo@85.174.183.252) |
| 2026-07-13 13:01:56 +0000 | <tomsmeding> | bwe: I guess you could still do [exa]'s suggestion, use `cabal --enable-profiling run mything -- +RTS -N -p` and look at the resulting .prof file, or post it here |
| 2026-07-13 13:04:43 +0000 | Guest6 | (~Guest6@147.161.130.251) |
| 2026-07-13 13:05:51 +0000 | <fp> | I was chatting with a vendor about benchmarking some realtime industrial Ethernet stuff. They said to buy their product, which runs 1000€. Woof. Well at least turns out there's a low end method I can use for ~20€ that only sacrifices bandwidth (but I still get like O(100Mb)) |
| 2026-07-13 13:06:05 +0000 | <bwe> | tomsmeding: fair enough, thank you for taking me until here! |
| 2026-07-13 13:06:19 +0000 | <fp> | So the moral is that throw hardware in and benchmarking can be hard *and* expensive |
| 2026-07-13 13:06:51 +0000 | <[exa]> | bwe: the "saving took x seconds" in the end is the latter phase of writing the whole result to sqlite? |
| 2026-07-13 13:07:03 +0000 | <[exa]> | btw how much of actual data/rows is it? |
| 2026-07-13 13:07:07 +0000 | <[exa]> | (roughly) |
| 2026-07-13 13:09:33 +0000 | <bwe> | [exa]: Yes, confirmed. It's writing the concatenated result lists at once. Those are ≈ 100 rows for the sample I ran for the most recent pastes. |
| 2026-07-13 13:11:16 +0000 | <[exa]> | because the first sample in here https://paste.tomsmeding.com/AqajXLxH : total elapsed time 3.78s, out of which I see like 3.66s saving time, is that right? |
| 2026-07-13 13:11:46 +0000 | <[exa]> | (same below, 5.765s vs 5.636s ish) |
| 2026-07-13 13:13:29 +0000 | <[exa]> | or is that expected because writing is interleaved with the parsing? |
| 2026-07-13 13:13:41 +0000 | <bwe> | [exa]: true. 41% for concurrently and 98% for serial is saving time. |
| 2026-07-13 13:13:54 +0000 | <bwe> | That's completely weird. |
| 2026-07-13 13:14:09 +0000 | <[exa]> | ok let's bisect more |
| 2026-07-13 13:14:47 +0000 | <[exa]> | can you temporarily remove the writing to sqlite to make sure it's not interfering in any way? (replace with `print` ;) ) |
| 2026-07-13 13:15:03 +0000 | <bwe> | [exa]: that's what I had on my mind, too. |
| 2026-07-13 13:15:34 +0000 | <[exa]> | I mean, there's like 3 or 4 other things that seem a bit off, so let's axe this down to an actual minimal reproducer |
| 2026-07-13 13:16:10 +0000 | <[exa]> | (and then we shall see) |
| 2026-07-13 13:18:25 +0000 | <bwe> | both variants now take ≈ 0.13 secs in total, each. it appears that lazy evaluation is an issue. |
| 2026-07-13 13:18:39 +0000 | tromp | (~textual@2001:1c00:340e:2700:70bb:dddb:6785:980d) (Quit: My iMac has gone to sleep. ZZZzzz…) |
| 2026-07-13 13:20:02 +0000 | <[exa]> | wait after you axed out the sql output it takes just 0.13s in total? |
| 2026-07-13 13:20:09 +0000 | <[exa]> | Are you still printing the parsed results? |
| 2026-07-13 13:20:18 +0000 | <[exa]> | (b/c if that's the case, SQLite is the issue) |
| 2026-07-13 13:21:02 +0000 | <bwe> | Fixed that with ST.print. Printing the results now. -- Now 5.7 sec total for serial, 9 sec total for concurrently. |
| 2026-07-13 13:21:12 +0000 | <[exa]> | ok phew :D |
| 2026-07-13 13:22:01 +0000 | <bwe> | I might abort mission and consider optimising the function itself :) |
| 2026-07-13 13:22:16 +0000 | <[exa]> | yeah well, can you do the profile? |
| 2026-07-13 13:22:43 +0000 | <[exa]> | not like it had to be optimized, but it would be best to clearly see if we're not parallelizing something that's super cheap to fix |
| 2026-07-13 13:25:16 +0000 | <[exa]> | also just curious, what does the parseAndPrepToUpdate actually do? |
| 2026-07-13 13:27:52 +0000 | <[exa]> | oh wow I was looking at where the forQuery came from and lo, it's our great forQuery from future selda :D |
| 2026-07-13 13:27:59 +0000 | <bwe> | it extracts / scrapes raw parts from html (using scalpel that itself uses html-parse) to a dto from which the raw parts get parsed using megaparsec, once that succeeds SaveMeToDB with some final sum type is created. That is being written to the db with an update. |
| 2026-07-13 13:28:03 +0000 | <bwe> | [exa]: coming from the future :0 |
| 2026-07-13 13:28:22 +0000 | <[exa]> | ok so I hope its not the part to blame :D |
| 2026-07-13 13:29:02 +0000 | <[exa]> | (how did you manage to make Stream an instance of MonadMask?) |
| 2026-07-13 13:29:31 +0000 | tromp | (~textual@2001:1c00:340e:2700:70bb:dddb:6785:980d) |
| 2026-07-13 13:31:13 +0000 | <merijn> | That question raises a bunch of spidey tingles |
| 2026-07-13 13:31:22 +0000 | <merijn> | I don't think Stream can be MonadMask? |
| 2026-07-13 13:32:10 +0000 | <bwe> | how to visualise the .prof file? I remember there was some simple web app for this? |
| 2026-07-13 13:32:19 +0000 | <[exa]> | just pastebin it |
| 2026-07-13 13:33:05 +0000 | <[exa]> | or you can run with -pj or something which gives you a json to upload to speedscope.app |
| 2026-07-13 13:34:34 +0000 | <merijn> | :O |
| 2026-07-13 13:34:43 +0000 | <merijn> | We have native speedscope support now? |
| 2026-07-13 13:34:45 +0000 | <merijn> | <3 |
| 2026-07-13 13:35:41 +0000 | <merijn> | Although be careful |
| 2026-07-13 13:35:50 +0000 | <merijn> | Foreign calls are hidden from profiling and thus speedscope |
| 2026-07-13 13:35:57 +0000 | <merijn> | Most notably SQLite operations |
| 2026-07-13 13:36:07 +0000 | <merijn> | (unless profiling was changed since I last looked) |
| 2026-07-13 13:36:36 +0000 | <[exa]> | that's the third thing to check for, just see whether the selects don't take too much time |
| 2026-07-13 13:36:57 +0000 | <[exa]> | (let's see the profile first) |
| 2026-07-13 13:38:09 +0000 | <merijn> | As someone whose written some horrific SQLite selects, you need to make it pretty bad before performance of those becomes a problem |
| 2026-07-13 13:38:48 +0000 | <bwe> | the profile is now with the deactivated db write: https://paste.tomsmeding.com/IydrsDQR |
| 2026-07-13 13:39:14 +0000 | <bwe> | (this is the head only as for now as the full 3.7 MiB didn't upload) |
| 2026-07-13 13:39:40 +0000 | <bwe> | [exa]: another prof with the sqlite db write? |
| 2026-07-13 13:40:12 +0000 | <merijn> | oh |
| 2026-07-13 13:40:19 +0000 | <merijn> | 61% in choice seems bad |
| 2026-07-13 13:40:43 +0000 | <merijn> | Is your parser maybe doing stupid amounts of backtracking? |
| 2026-07-13 13:40:48 +0000 | bwe | agrees. |
| 2026-07-13 13:40:50 +0000 | <merijn> | What this code even doing? :) |
| 2026-07-13 13:41:12 +0000 | <merijn> | Also, 20% in parserFactory seems like you might get some speed up by caching the parser creation? |
| 2026-07-13 13:41:23 +0000 | <merijn> | I mean, it seems like everything is using the same parser, right? |
| 2026-07-13 13:41:34 +0000 | <bwe> | merijn: Exactly. |
| 2026-07-13 13:41:52 +0000 | <bwe> | Wait. Let me double-check. |
| 2026-07-13 13:41:53 +0000 | <[exa]> | merijn: might be mislabeled because there's lambdas in there |
| 2026-07-13 13:42:18 +0000 | <[exa]> | bwe: anyway I guess time to look at parser code |
| 2026-07-13 13:42:21 +0000 | <[exa]> | :D |
| 2026-07-13 13:44:11 +0000 | <[exa]> | bwe: completely out of good sports practice, try compiling with -fexpose-all-unfoldings -fspecialise-aggressively -O2 |
| 2026-07-13 13:44:31 +0000 | <[exa]> | just to make sure we're not looking at inliner slacking where it should not |
| 2026-07-13 13:47:35 +0000 | <bwe> | merijn: Confirmed. I build one parser. So I gonna just build it once on load and cache it in an IORef. Then, when I use it, I pull it in from the Reader. |
| 2026-07-13 13:49:02 +0000 | AlexZenon | (~alzenon@85.174.183.252) |
| 2026-07-13 13:50:17 +0000 | <bwe> | [exa]: with or without profiling? |
| 2026-07-13 13:54:29 +0000 | jle` | (~jle`@2603:8001:3b00:11::1fae) (Ping timeout: 245 seconds) |
| 2026-07-13 13:55:19 +0000 | Alex_delenda_est | (~al_test@85.174.183.252) |
| 2026-07-13 13:55:36 +0000 | jle` | (~jle`@2603:8001:3b00:11::1fae) jle` |
| 2026-07-13 13:57:05 +0000 | <[exa]> | bwe: without, this is just a check for the case that the optimizer e.g. didn't see something that makes stuff allocate too much or so |
| 2026-07-13 13:58:05 +0000 | <[exa]> | on the other webby project I have like 20x performance difference, likely because it's not choosing to specialize for concrete instances across modules by default |
| 2026-07-13 14:00:39 +0000 | <bwe> | merijn: choice is called by Text.HTML.Parser.parseTokens (html-parse) eventually in https://hackage-content.haskell.org/package/html-parse-0.2.2.0/docs/src/Text.HTML.Parser.html#data… |
| 2026-07-13 14:01:39 +0000 | <bwe> | [exa]: 16 secs for the concurrent variant with your bad sports practices |
| 2026-07-13 14:04:14 +0000 | <bwe> | merijn: any alternative to https://hackage.haskell.org/package/attoparsec-0.14.4/docs/Data-Attoparsec-Combinator.html#v:choice that is faster? |
| 2026-07-13 14:15:59 +0000 | puke | (~puke@user/puke) (Read error: Connection reset by peer) |
| 2026-07-13 14:16:17 +0000 | puke | (~puke@user/puke) puke |
| 2026-07-13 14:19:15 +0000 | Wygulmage | (~Wygulmage@user/Wygulmage) Wygulmage |
| 2026-07-13 14:26:59 +0000 | <[exa]> | bwe: wait what, it got _slower_ ? |
| 2026-07-13 14:27:47 +0000 | <[exa]> | bwe: anyway `choice` is generally unavoidable _but_ you usually want to have the branches commit as early as possible (i.e., no `try`) |
| 2026-07-13 14:32:55 +0000 | CiaoSen | (~Jura@2a02:3100:59f8:a800:4e50:ddff:fe9b:8922) (Ping timeout: 276 seconds) |
| 2026-07-13 14:33:18 +0000 | <merijn> | [exa]: attoparsec auto backtracks |
| 2026-07-13 14:33:40 +0000 | <merijn> | try in attoparsec is a no-op for compatibility with parsec |
| 2026-07-13 14:34:02 +0000 | whereiseveryone | (206ba86c98@2a03:6000:1812:100::2e4) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | carbolymer | (carbolymer@delirium.systems) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | dsal | (sid13060@id-13060.lymington.irccloud.com) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | cbarrett | (sid192934@id-192934.helmsley.irccloud.com) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | hook54321 | (sid149355@user/hook54321) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | T_S_______ | (sid501726@id-501726.uxbridge.irccloud.com) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | megeve | (sid727922@id-727922.lymington.irccloud.com) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | jackdk | (sid373013@cssa/life/jackdk) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | Fangs | (sid141280@id-141280.hampstead.irccloud.com) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | degraafk | (sid71464@id-71464.lymington.irccloud.com) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | luke | (~luke@user/luke) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | glguy | (glguy@libera/staff/glguy) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | rensenwxre | (~fwam@user/fwam) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | lxsameer | (~lxsameer@Serene/lxsameer) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | noctux1 | (~noctux@user/noctux) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | apache2 | (apache2@anubis.0x90.dk) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | mangoiv | (~mangoiv@user/mangoiv) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | APic | (apic@chiptune.apic.name) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | darkling | (~darkling@2001-ba8-1f1-f0e6-0-0-0-2.autov6rev.bitfolk.space) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | flukiluke | (~m-7humut@2603:c023:c000:6c7e:8945:ad24:9113:a962) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | bwe | (~bwe@2a01:4f8:1c1c:4878::2) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | xnyhps | (~xnyhps@s.xnyhps.nl) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | eugenrh | (~eugenrh@user/eugenrh) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | uint64_t | (ec7b9bb9f3@2a03:6000:1812:100::139b) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | shreyasminocha | (51fdc93eda@user/shreyasminocha) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | filwisher | (2e6936c793@2a03:6000:1812:100::170) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | b0o | (0e4a0bf4c9@2a03:6000:1812:100::1bf) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | TimWolla | (~timwolla@2a01:4f8:150:6153:beef::6667) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | reyu | (~reyu@znc.reyuzenfold.com) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | koolazer | (~koo@user/koolazer) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | catties | (~catties@user/meow/catties) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | mmaruseacph2 | (~mihai@mihai.page) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | Guest9151 | (bnc4free@here.and.ready-to.party) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | Aleksejs | (~Aleksejs@107.170.21.106) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | nonzen_ | (~nonzen@user/nonzen) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | Putonlalla | (~Putonlall@it-cyan.it.jyu.fi) (*.net *.split) |
| 2026-07-13 14:34:02 +0000 | chiselfuse | (~chiselfus@user/chiselfuse) (*.net *.split) |
| 2026-07-13 14:34:06 +0000 | glguy | (glguy@libera/staff/glguy) glguy |
| 2026-07-13 14:34:11 +0000 | T_S_______ | (sid501726@id-501726.uxbridge.irccloud.com) |
| 2026-07-13 14:34:12 +0000 | carbolymer | (carbolymer@delirium.systems) carbolymer |
| 2026-07-13 14:34:12 +0000 | shreyasminocha | (51fdc93eda@user/shreyasminocha) shreyasminocha |
| 2026-07-13 14:34:12 +0000 | rensenwxre | (~fwam@user/fwam) fwam |
| 2026-07-13 14:34:12 +0000 | uint64_t | (ec7b9bb9f3@2a03:6000:1812:100::139b) k_hachig |
| 2026-07-13 14:34:13 +0000 | Aleksejs | (~Aleksejs@107.170.21.106) aleksejs |
| 2026-07-13 14:34:13 +0000 | TimWolla | (~timwolla@2a01:4f8:150:6153:beef::6667) TimWolla |
| 2026-07-13 14:34:14 +0000 | <merijn> | So stupidly written attoparsec parsers can incur A LOT of backtracking cost |
| 2026-07-13 14:34:14 +0000 | dsal | (sid13060@id-13060.lymington.irccloud.com) dsal |
| 2026-07-13 14:34:15 +0000 | whereiseveryone | (206ba86c98@2a03:6000:1812:100::2e4) jgart |
| 2026-07-13 14:34:15 +0000 | filwisher | (2e6936c793@2a03:6000:1812:100::170) |
| 2026-07-13 14:34:15 +0000 | degraafk | (sid71464@id-71464.lymington.irccloud.com) degraafk |
| 2026-07-13 14:34:16 +0000 | luke | (~luke@user/luke) luke |
| 2026-07-13 14:34:16 +0000 | xnyhps | (~xnyhps@s.xnyhps.nl) |
| 2026-07-13 14:34:17 +0000 | apache2 | (apache2@46.101.137.181) apache2 |
| 2026-07-13 14:34:17 +0000 | b0o | (0e4a0bf4c9@2a03:6000:1812:100::1bf) b0o |
| 2026-07-13 14:34:17 +0000 | cbarrett | (sid192934@2a03:5180:f:1::2:f1a6) cbarrett |
| 2026-07-13 14:34:18 +0000 | darkling | (~darkling@savella.carfax.org.uk) |
| 2026-07-13 14:34:18 +0000 | Fangs | (sid141280@id-141280.hampstead.irccloud.com) Fangs |
| 2026-07-13 14:34:18 +0000 | lxsameer | (~lxsameer@Serene/lxsameer) lxsameer |
| 2026-07-13 14:34:18 +0000 | nonzen | (~nonzen@user/nonzen) nonzen |
| 2026-07-13 14:34:19 +0000 | mangoiv | (~mangoiv@2a01:4f9:c012:6c0e::) |
| 2026-07-13 14:34:19 +0000 | mmaruseacph2 | (~mihai@mihai.page) mmaruseacph2 |
| 2026-07-13 14:34:20 +0000 | koolazer | (~koo@user/koolazer) koolazer |
| 2026-07-13 14:34:20 +0000 | jackdk | (sid373013@cssa/life/jackdk) jackdk |
| 2026-07-13 14:34:20 +0000 | eugenrh | (~eugenrh@user/eugenrh) eugenrh |
| 2026-07-13 14:34:21 +0000 | reyu | (~reyu@znc.reyuzenfold.com) Reyu |
| 2026-07-13 14:34:21 +0000 | noctuks | (~noctux@user/noctux) noctux |
| 2026-07-13 14:34:22 +0000 | catties | (~catties@user/meow/catties) catties |
| 2026-07-13 14:34:22 +0000 | flukiluke | (~m-7humut@2603:c023:c000:6c7e:8945:ad24:9113:a962) flukiluke |
| 2026-07-13 14:34:24 +0000 | megeve | (sid727922@id-727922.lymington.irccloud.com) megeve |
| 2026-07-13 14:34:29 +0000 | Putonlalla | (~Putonlall@it-cyan.it.jyu.fi) Tuplanolla |
| 2026-07-13 14:34:34 +0000 | chiselfuse | (~chiselfus@user/chiselfuse) chiselfuse |
| 2026-07-13 14:35:22 +0000 | hook54321 | (sid149355@user/hook54321) hook54321 |
| 2026-07-13 14:35:46 +0000 | <[exa]> | merijn: wait what?! |
| 2026-07-13 14:36:08 +0000 | <[exa]> | I always thought they gutted that possibility out to get speed |
| 2026-07-13 14:37:26 +0000 | <merijn> | [exa]: https://hackage.haskell.org/package/attoparsec-0.14.4/docs/Data-Attoparsec-ByteString.html#v:try |
| 2026-07-13 14:37:30 +0000 | <[exa]> | (...this information comes to me after writing like 2 production parsers in it and not even noticing) |
| 2026-07-13 14:37:56 +0000 | <[exa]> | (the only major perf issue I had was converting `many` to `scan`s) |
| 2026-07-13 14:38:02 +0000 | <[exa]> | guess I'm lucky |
| 2026-07-13 14:38:04 +0000 | crtschin | (c0d655fd40@user/crtschin) (Ping timeout: 252 seconds) |
| 2026-07-13 14:38:11 +0000 | <merijn> | or you write good code :p |
| 2026-07-13 14:38:16 +0000 | tromp | (~textual@2001:1c00:340e:2700:70bb:dddb:6785:980d) (Quit: My iMac has gone to sleep. ZZZzzz…) |
| 2026-07-13 14:38:23 +0000 | <merijn> | Good parsers shouldn't randomly backtrack anyway :p |
| 2026-07-13 14:38:50 +0000 | <merijn> | [exa]: If you branch on, for example, tokens the backtracking isn't too costly |
| 2026-07-13 14:38:52 +0000 | crtschin | (c0d655fd40@user/crtschin) crtschin |
| 2026-07-13 14:39:12 +0000 | APic | (apic@chiptune.apic.name) APic |
| 2026-07-13 14:39:23 +0000 | <merijn> | [exa]: Suppose your 5 choices deep and every choice is just dispatch between 5 static tokens, you just need to check, like, 25 static tokens for the backtrack to fail in all branches |
| 2026-07-13 14:39:37 +0000 | machinedgod | (~machinedg@d108-173-95-19.abhsia.telus.net) machinedgod |
| 2026-07-13 14:39:46 +0000 | <merijn> | [exa]: The real problem is if you write choice branches that need to parse *a lot* before deciding they should fail |
| 2026-07-13 14:39:52 +0000 | <[exa]> | nah I killed all deep choices |
| 2026-07-13 14:40:04 +0000 | <merijn> | [exa]: Right, because those are a nightmare to maintain, write, test, etc. |
| 2026-07-13 14:40:16 +0000 | <merijn> | And then default backtracking isn't an issue |
| 2026-07-13 14:40:23 +0000 | <[exa]> | luckily the language in question kinda depends on a separate lexing pass |
| 2026-07-13 14:40:54 +0000 | <[exa]> | who'd say that's a maintainability-improving feature |
| 2026-07-13 14:41:11 +0000 | <merijn> | Literally everyone who's written a parser? :p |
| 2026-07-13 14:41:43 +0000 | [exa] | runs and hides |
| 2026-07-13 14:42:10 +0000 | <[exa]> | oh noes looks like bwe got pwned by the netsplit |
| 2026-07-13 14:47:22 +0000 | jinsun | (bnc4free@here.and.ready-to.party) |
| 2026-07-13 14:47:34 +0000 | danz37027 | (~danza@user/danza) (Remote host closed the connection) |
| 2026-07-13 14:47:45 +0000 | jinsun | Guest9130 |
| 2026-07-13 14:49:36 +0000 | Guest6 | (~Guest6@147.161.130.251) (Quit: Client closed) |
| 2026-07-13 14:58:13 +0000 | chromoblob | (~chromoblo@user/chromob1ot1c) (Ping timeout: 240 seconds) |
| 2026-07-13 14:58:15 +0000 | picnoir | (~picnoir@about/aquilenet/vodoo/NinjaTrappeur) (Quit: WeeChat 4.9.3) |
| 2026-07-13 14:58:20 +0000 | bwe | (~bwe@2a01:4f8:1c1c:4878::2) bwe |
| 2026-07-13 14:59:01 +0000 | <bwe> | [exa]: I haven't understood try and choice fully yet to tamper around with it, honestly. |
| 2026-07-13 14:59:22 +0000 | picnoir | (~picnoir@about/aquilenet/vodoo/NinjaTrappeur) NinjaTrappeur |
| 2026-07-13 14:59:56 +0000 | <[exa]> | bwe: can you have a look at the profile (or upload it somewhere so that we can) to check which exact `choice` takes most time there? |
| 2026-07-13 15:02:43 +0000 | <bwe> | [exa]: this https://hackage-content.haskell.org/package/html-parse-0.2.2.0/docs/src/Text.HTML.Parser.html#data… |
| 2026-07-13 15:04:29 +0000 | fp | (~Thunderbi@130.233.70.229) (Ping timeout: 245 seconds) |
| 2026-07-13 15:06:41 +0000 | <[exa]> | bwe: how many MBs of HTML does it chew through? (roughly) |
| 2026-07-13 15:09:54 +0000 | ouilemur | (~jgmerritt@user/ouilemur) (Ping timeout: 245 seconds) |
| 2026-07-13 15:13:28 +0000 | ouilemur | (~jgmerritt@user/ouilemur) ouilemur |
| 2026-07-13 15:15:26 +0000 | tromp | (~textual@2001:1c00:340e:2700:70bb:dddb:6785:980d) |
| 2026-07-13 15:20:32 +0000 | chromoblob | (~chromoblo@user/chromob1ot1c) chromoblob\0 |
| 2026-07-13 15:23:18 +0000 | <bwe> | [exa]: depends, I've got some that are 2.5 MiB. Which is insane if you think about it. |
| 2026-07-13 15:26:18 +0000 | <bwe> | merijn: Now I've got where! I've cached the built parser and end up with concurrently 2.5 sec while serial is 5.9 sec total |
| 2026-07-13 15:27:20 +0000 | <bwe> | That might also explain why concurrent variant was slower because it had most work with building the parser (and different threads couldn't benefit from each other). |
| 2026-07-13 15:29:07 +0000 | ouilemur | (~jgmerritt@user/ouilemur) (Ping timeout: 264 seconds) |
| 2026-07-13 15:33:34 +0000 | ouilemur | (~jgmerritt@user/ouilemur) ouilemur |
| 2026-07-13 15:37:02 +0000 | <[exa]> | oh there we go |
| 2026-07-13 15:37:04 +0000 | <[exa]> | cool |
| 2026-07-13 15:38:18 +0000 | emilym | (~Thunderbi@user/emilym) emilym |
| 2026-07-13 15:41:28 +0000 | chele | (~chele@user/chele) (Remote host closed the connection) |
| 2026-07-13 15:42:31 +0000 | emilym | (~Thunderbi@user/emilym) (Ping timeout: 240 seconds) |
| 2026-07-13 15:44:14 +0000 | <tomsmeding> | wait why does that have anything to do with concurrent vs serial |
| 2026-07-13 15:44:33 +0000 | <tomsmeding> | if you get a speedup from caching the parser, surely you get the same speedup if you apply the parser sequentially? |
| 2026-07-13 15:44:49 +0000 | <tomsmeding> | bwe: ^ |
| 2026-07-13 15:48:03 +0000 | <[exa]> | I guess we're getting there |
| 2026-07-13 15:49:03 +0000 | <[exa]> | yeah in short, what does buildParser do? (IO?) |
| 2026-07-13 15:50:43 +0000 | chromoblob | (~chromoblo@user/chromob1ot1c) (Ping timeout: 264 seconds) |
| 2026-07-13 15:51:42 +0000 | chromoblob | (~chromoblo@user/chromob1ot1c) chromoblob\0 |
| 2026-07-13 15:58:26 +0000 | lbseale | (~quassel@user/ep1ctetus) (Ping timeout: 265 seconds) |
| 2026-07-13 16:00:05 +0000 | <tomsmeding> | although <2.5x speedup for 12 cores is underwhelming still |
| 2026-07-13 16:00:39 +0000 | chromoblob | (~chromoblo@user/chromob1ot1c) (Read error: Connection reset by peer) |
| 2026-07-13 16:01:02 +0000 | lbseale | (~quassel@user/ep1ctetus) ep1ctetus |
| 2026-07-13 16:01:04 +0000 | chromoblob | (~chromoblo@user/chromob1ot1c) chromoblob\0 |
| 2026-07-13 16:04:53 +0000 | tromp | (~textual@2001:1c00:340e:2700:70bb:dddb:6785:980d) (Quit: My iMac has gone to sleep. ZZZzzz…) |
| 2026-07-13 16:04:54 +0000 | <EvanR> | you heard of embarrassingly parallel? well this is parallelly embarassing |
| 2026-07-13 16:05:10 +0000 | chromoblob | (~chromoblo@user/chromob1ot1c) (Ping timeout: 245 seconds) |
| 2026-07-13 16:05:31 +0000 | chromoblob | (~chromoblo@user/chromob1ot1c) chromoblob\0 |
| 2026-07-13 16:05:48 +0000 | tromp | (~textual@2001:1c00:340e:2700:70bb:dddb:6785:980d) |
| 2026-07-13 16:07:54 +0000 | chromoblob | (~chromoblo@user/chromob1ot1c) (Read error: Connection reset by peer) |
| 2026-07-13 16:08:39 +0000 | chromoblob | (~chromoblo@user/chromob1ot1c) chromoblob\0 |
| 2026-07-13 16:25:48 +0000 | <gentauro> | anybody here who worked with `haskell-language-server`? If yes, what's the reason to (when using) CLI to output to `stderr` in a non-structured way? |
| 2026-07-13 16:27:59 +0000 | marinelli | (~weechat@gateway/tor-sasl/marinelli) (Remote host closed the connection) |
| 2026-07-13 16:28:05 +0000 | <[exa]> | stderr is typically for unstructured messages to user, right? |
| 2026-07-13 16:28:10 +0000 | <[exa]> | (kindof "out of band" data) |
| 2026-07-13 16:31:44 +0000 | pavonia | (~user@user/siracusa) (Quit: Bye!) |
| 2026-07-13 16:31:57 +0000 | marinelli | (~weechat@gateway/tor-sasl/marinelli) marinelli |
| 2026-07-13 16:32:43 +0000 | <gentauro> | [exa]: yeah, but it's a CLI tool. I'm guessing the normal output should be sent to `stdout` |
| 2026-07-13 16:32:58 +0000 | <gentauro> | by sending to `stderr` pipelines think it's failing and … |
| 2026-07-13 16:33:00 +0000 | <gentauro> | :-\ |
| 2026-07-13 16:33:26 +0000 | gentauro | just to clarify. There is nothing sent to `stdout` when running the tool as CLI |
| 2026-07-13 16:34:37 +0000 | <tomsmeding> | gentauro: isn't the point of HLS to talk HTTP-ish on stdout? |
| 2026-07-13 16:35:12 +0000 | <tomsmeding> | (and since when is the appearance of text on stderr considered failure?) |
| 2026-07-13 16:36:28 +0000 | <gentauro> | tomsmeding: -> https://haskell-language-server.readthedocs.io/en/stable/troubleshooting.html#reproducing-failures… |
| 2026-07-13 16:36:30 +0000 | <tomsmeding> | if you mean the debug output printed when you run `haskell-language-server` without further arguments, well, that's debug output and probably implemented as an extension of the stuff it normally already prints on stderr during normal --lsp operation |
| 2026-07-13 16:36:49 +0000 | <tomsmeding> | right, then ^ I guess |
| 2026-07-13 16:37:01 +0000 | <tomsmeding> | non-`--lsp` operation is debug mode |
| 2026-07-13 16:37:54 +0000 | <tomsmeding> | detecting errors in pipelines would normally use the exit code, not whether text has been produced on stderr, I think |
| 2026-07-13 16:38:17 +0000 | <gentauro> | tomsmeding> non-`--lsp` operation is debug mode. Oh, got it |
| 2026-07-13 16:38:45 +0000 | <gentauro> | makes sense now as I'm seeing a bunch of `debug` statements even though I have limited to `Info` |
| 2026-07-13 16:40:18 +0000 | <tomsmeding> | like, it's meant to be a language server :p |
| 2026-07-13 16:40:54 +0000 | <tomsmeding> | the argument-less mode is for diagnosing issues that are hard to check when it's being used as a language server |
| 2026-07-13 16:41:19 +0000 | <tomsmeding> | in particular, the argument-less mode tries to load all .hs files it can find in the current directory, I think, regardless of what project they are in or not |
| 2026-07-13 16:41:31 +0000 | <gentauro> | tomsmeding: yeah, but sometimes you want to feed it a project and a single file (could be HUGE :P) and see why it's crashin/freezing :P |
| 2026-07-13 16:41:39 +0000 | <tomsmeding> | which means that if you have some cruft folder lying around, it will try to load that and (probably ) error out |
| 2026-07-13 16:41:56 +0000 | <tomsmeding> | right |
| 2026-07-13 16:42:11 +0000 | <tomsmeding> | well, I guess the underlying point is "stderr output does not mean an error occurred" :p |
| 2026-07-13 16:42:26 +0000 | <tomsmeding> | so if you have a tool that assumes that, maybe that tool is the one being unconventional |
| 2026-07-13 16:42:33 +0000 | <gentauro> | tomsmeding: do you even do Azure pipelines? :P |
| 2026-07-13 16:42:36 +0000 | <tomsmeding> | no |
| 2026-07-13 16:42:41 +0000 | <tomsmeding> | and apparently I'm happy I don't |
| 2026-07-13 16:42:52 +0000 | tzh | (~tzh@c-76-115-131-146.hsd1.or.comcast.net) |
| 2026-07-13 16:42:52 +0000 | <tomsmeding> | stick 2>&1 after it or something |
| 2026-07-13 16:42:55 +0000 | <gentauro> | I wish I could say the same :D |
| 2026-07-13 16:43:07 +0000 | <tomsmeding> | answer stupid tools with stupid workarounds |
| 2026-07-13 16:43:07 +0000 | <gentauro> | but, somebody has to do the dirty work and pay taxes so … :P |
| 2026-07-13 16:43:30 +0000 | <tomsmeding> | there's probably other parts of the job that are fun |
| 2026-07-13 16:43:57 +0000 | Googulator | (~Googulato@193-226-241-68.pool.digikabel.hu) (Quit: Client closed) |
| 2026-07-13 16:43:58 +0000 | <gentauro> | in the ages of AI-slop, nothing is fun anymore :( |
| 2026-07-13 16:44:24 +0000 | Googulator | (~Googulato@193-226-241-68.pool.digikabel.hu) |
| 2026-07-13 16:44:41 +0000 | <tomsmeding> | for curiosity, why are you running HLS in CI? |
| 2026-07-13 16:44:49 +0000 | <tomsmeding> | (assuming "azure pipeline" means "CI") |
| 2026-07-13 16:45:11 +0000 | <tomsmeding> | gentauro: can relate, though not everyone uses LLMs to produce slop |
| 2026-07-13 16:45:58 +0000 | <gentauro> | tomsmeding: is somehow a "cheap" type-checker ;) |
| 2026-07-13 16:46:37 +0000 | <tomsmeding> | but if you're in a place where slop is considered great productivity, then I'm sorry for you, hang in there; the field is rapidly evolving, including people's sentiments, so the world may well look different again in a year or two |
| 2026-07-13 16:46:45 +0000 | <tomsmeding> | gentauro: interesting, what about ghci? |
| 2026-07-13 16:47:51 +0000 | <gentauro> | the problem with AI-slop is the problem that always exited: Those who case (people who for example are here on a Monday at 19:00 ish) and those who don't give a damn |
| 2026-07-13 16:48:35 +0000 | <gentauro> | by giving them AI-sloop tools, they are somehow superproductive generating even more garbage, which is backed up my management (do NOT dare to reject on basis of quality, readability, …) |
| 2026-07-13 16:49:08 +0000 | <gentauro> | it's the world we live in … until `tokens` will become to expensive and then, all the "vibe coders" will have a bad time trying to code again |
| 2026-07-13 16:49:27 +0000 | gentauro | might be #haskell-offtop :) |
| 2026-07-13 16:51:20 +0000 | tromp | (~textual@2001:1c00:340e:2700:70bb:dddb:6785:980d) (Quit: My iMac has gone to sleep. ZZZzzz…) |
| 2026-07-13 16:51:55 +0000 | <tomsmeding> | well, tokens are getting more expensive; stay optimistic :) |
| 2026-07-13 16:52:09 +0000 | <gentauro> | xD |
| 2026-07-13 16:55:25 +0000 | arandombit | (~arandombi@user/arandombit) (Remote host closed the connection) |
| 2026-07-13 16:57:07 +0000 | <EvanR> | replace the management with AI then |
| 2026-07-13 17:00:01 +0000 | chexum | (~quassel@gateway/tor-sasl/chexum) (Ping timeout: 252 seconds) |
| 2026-07-13 17:00:24 +0000 | chexum | (~quassel@gateway/tor-sasl/chexum) chexum |
| 2026-07-13 17:05:31 +0000 | arandombit | (~arandombi@2603:7000:4600:ffbe:2463:4999:2446:85d6) |
| 2026-07-13 17:05:31 +0000 | arandombit | (~arandombi@2603:7000:4600:ffbe:2463:4999:2446:85d6) (Changing host) |
| 2026-07-13 17:05:31 +0000 | arandombit | (~arandombi@user/arandombit) arandombit |
| 2026-07-13 17:10:52 +0000 | spew | (~spew@user/spew) spew |
| 2026-07-13 17:15:29 +0000 | ski | (~ski@31-208-30-136.cust.bredband2.com) (Ping timeout: 252 seconds) |
| 2026-07-13 17:17:19 +0000 | ski | (~ski@31-208-28-132.cust.bredband2.com) |
| 2026-07-13 17:26:58 +0000 | ft | (~ft@p508dbe16.dip0.t-ipconnect.de) ft |
| 2026-07-13 17:50:56 +0000 | takuan | (~takuan@d8D86B996.access.telenet.be) |
| 2026-07-13 17:51:00 +0000 | takuan | (~takuan@d8D86B996.access.telenet.be) (Client Quit) |
| 2026-07-13 17:51:36 +0000 | m4rv | (~m4rv@p200300ca8f01e000c3fa7c546c865f62.dip0.t-ipconnect.de) |
| 2026-07-13 17:54:30 +0000 | Lord_of_Life | (~Lord@user/lord-of-life/x-2819915) (Quit: Laa shay'a waqi'un moutlaq bale kouloun moumkine) |
| 2026-07-13 17:56:28 +0000 | Lord_of_Life | (~Lord@user/lord-of-life/x-2819915) Lord_of_Life |
| 2026-07-13 17:59:29 +0000 | Lord_of_Life | (~Lord@user/lord-of-life/x-2819915) (Excess Flood) |
| 2026-07-13 18:01:43 +0000 | <[exa]> | gentauro: tbh pipelines that interpret stderr presence as error have a serious issue |
| 2026-07-13 18:01:48 +0000 | m4rv | (~m4rv@p200300ca8f01e000c3fa7c546c865f62.dip0.t-ipconnect.de) (Changing host) |
| 2026-07-13 18:01:48 +0000 | m4rv | (~m4rv@user/m4rv) m4rv |
| 2026-07-13 18:02:12 +0000 | m4rv | (~m4rv@user/m4rv) (Quit: Client closed) |
| 2026-07-13 18:02:33 +0000 | m4rv | (~m4rv@p200300ca8f01e000c3fa7c546c865f62.dip0.t-ipconnect.de) |
| 2026-07-13 18:02:42 +0000 | m4rv | (~m4rv@p200300ca8f01e000c3fa7c546c865f62.dip0.t-ipconnect.de) (Write error: Broken pipe) |
| 2026-07-13 18:02:57 +0000 | m4rv | (~m4rv@p200300ca8f01e000c3fa7c546c865f62.dip0.t-ipconnect.de) |
| 2026-07-13 18:03:17 +0000 | m4rv | (~m4rv@p200300ca8f01e000c3fa7c546c865f62.dip0.t-ipconnect.de) (Client Quit) |
| 2026-07-13 18:03:32 +0000 | Wygulmage | (~Wygulmage@user/Wygulmage) (Ping timeout: 245 seconds) |
| 2026-07-13 18:17:16 +0000 | <tomsmeding> | [exa]: I have a suspicion that they already know that their software has serious issues |
| 2026-07-13 18:20:57 +0000 | Lord_of_Life | (~Lord@user/lord-of-life/x-2819915) Lord_of_Life |
| 2026-07-13 18:24:18 +0000 | tromp | (~textual@2001:1c00:340e:2700:70bb:dddb:6785:980d) |
| 2026-07-13 18:24:52 +0000 | Lord_of_Life | (~Lord@user/lord-of-life/x-2819915) (Excess Flood) |
| 2026-07-13 18:27:19 +0000 | Lord_of_Life | (~Lord@user/lord-of-life/x-2819915) Lord_of_Life |
| 2026-07-13 18:28:25 +0000 | sim590_ | (~sim590@209-15-184-6.resi.cgocable.ca) sim590 |
| 2026-07-13 18:28:30 +0000 | sim590 | (~sim590@209-15-184-6.resi.cgocable.ca) (Read error: Connection reset by peer) |
| 2026-07-13 18:29:02 +0000 | lisbeths | (uid135845@id-135845.lymington.irccloud.com) (Quit: Connection closed for inactivity) |
| 2026-07-13 18:33:33 +0000 | <[exa]> | tomsmeding: I was wondering who exactly you meant but luckily the statement applies generally |
| 2026-07-13 18:37:14 +0000 | target_i | (~target_i@user/target-i/x-6023099) target_i |
| 2026-07-13 18:37:24 +0000 | sgillespie | (~sgillespi@user/sgillespie) (Remote host closed the connection) |
| 2026-07-13 18:40:24 +0000 | <tomsmeding> | [exa]: lol |
| 2026-07-13 18:40:35 +0000 | <tomsmeding> | I meant gentauro, given that the software in question is azure pipelines |
| 2026-07-13 18:45:00 +0000 | <gentauro> | [exa]: it's the default behaviour of Azure :| |
| 2026-07-13 18:51:59 +0000 | <EvanR> | not allowed to speak your mind on stderr? What is this tyranny |
| 2026-07-13 18:57:21 +0000 | schuelermine | (~Thunderbi@user/schuelermine) schuelermine |
| 2026-07-13 18:58:57 +0000 | <monochrom> | Or rather, not allowed to criticize LLM outputs. |
| 2026-07-13 19:00:37 +0000 | <tomsmeding> | we only criticise LLMs on stderr here |
| 2026-07-13 19:01:11 +0000 | <monochrom> | haha |
| 2026-07-13 19:10:11 +0000 | <bwe> | [exa]: https://paste.tomsmeding.com/VFVYOAOe <-- here is what the buildParser does |
| 2026-07-13 19:11:11 +0000 | <bwe> | tomsmeding: I agree but 2.5x speedup is an improvement (while since it has been a worsening) |
| 2026-07-13 19:14:44 +0000 | gf3_ | (~gf3@142.188.71.198) (Remote host closed the connection) |
| 2026-07-13 19:16:08 +0000 | gf3 | (~gf3@user/gf3) gf3 |
| 2026-07-13 19:16:16 +0000 | <[exa]> | bwe: ah so basically it makes a huge `choice` 2 times |
| 2026-07-13 19:16:53 +0000 | <[exa]> | bwe: oh noes, why is there foldL on line 4? (why not `foldr` or `choice` right away?) |
| 2026-07-13 19:19:05 +0000 | <bwe> | [exa]: I didn't know better |
| 2026-07-13 19:19:37 +0000 | <bwe> | [exa]: |
| 2026-07-13 19:19:58 +0000 | <bwe> | [exa]: why does foldr perform better? I know there's a difference but can't recall. |
| 2026-07-13 19:21:36 +0000 | <[exa]> | the <|> is trying the left possibility first, so it's better if not's parenthesized super deeply |
| 2026-07-13 19:21:52 +0000 | Pozyomka | (~pyon@user/pyon) (Remote host closed the connection) |
| 2026-07-13 19:21:56 +0000 | Pozyomka_ | (~pyon@user/pyon) pyon |
| 2026-07-13 19:22:00 +0000 | <[exa]> | (anyway I think that you can replace the whole fold with just `choice`) |
| 2026-07-13 19:22:34 +0000 | <[exa]> | s/if not's/if it's not/ whoops |
| 2026-07-13 19:25:43 +0000 | <bwe> | Changed it to choice. Didn't alter the execution time (which makes sense as I build the parser now once and cache it). |
| 2026-07-13 19:31:58 +0000 | <[exa]> | ok for the rest I guess the main issue is that it simply backtracks too much |
| 2026-07-13 19:34:19 +0000 | <[exa]> | bwe: hey as I'm reading this, isn't it kinda doing the job of aho-corasick multistring matcher? |
| 2026-07-13 19:36:33 +0000 | <[exa]> | (A-C algorithm: you throw it a bazillion needles, it prepares stuff in O(|needles|) and then matches all of them at once in a string of length n in linear time) |
| 2026-07-13 19:38:39 +0000 | bwe | reads https://github.com/channable/alfred-margaret |
| 2026-07-13 19:40:05 +0000 | <[exa]> | a smaller gun might be sufficient https://hackage.haskell.org/package/AhoCorasick |
| 2026-07-13 19:40:23 +0000 | <bwe> | [exa]: Yes, it actually handles many variants for one representative name. But then it differentiates between different variants of hits. |
| 2026-07-13 19:43:42 +0000 | <[exa]> | ok that might not help much |
| 2026-07-13 19:44:01 +0000 | <[exa]> | still I guess pre-chewing the input strings into some kind of trie and then reducing choices to that trie only could help |
| 2026-07-13 19:44:29 +0000 | <[exa]> | how much of the alternatives to be parsed do you have there? |
| 2026-07-13 19:45:15 +0000 | <[exa]> | if it's over 100 I'd say there's no chance that it's slower with a trie |
| 2026-07-13 19:45:27 +0000 | <[exa]> | esp. if prefixes are shared |
| 2026-07-13 19:45:57 +0000 | <bwe> | [exa]: usually 6-12 alternatives |
| 2026-07-13 19:47:02 +0000 | <[exa]> | okay that's not that much then |
| 2026-07-13 19:47:56 +0000 | <bwe> | [exa]: so the tree idea is to already exclude many variants from the input side instead of only excluding by matching on the pattern side? |
| 2026-07-13 19:48:41 +0000 | <[exa]> | yeah if you'd have variants that start with same letter(s), the idea is to only match that initial letter once and then pick only from the relevant variants |
| 2026-07-13 19:51:05 +0000 | <[exa]> | iirc there are some parser libraries that would do this for you (at least partially) iirc not attoparsec |
| 2026-07-13 19:56:02 +0000 | <bwe> | [exa]: that's neat. you very disciplined ask yourself what are you doing but don't need to be doing on a micro level. |
| 2026-07-13 19:56:46 +0000 | <bwe> | [exa]: scalpel uses html-parse that uses attoparsec. I use megaparsec to parse the scraped html fragments, though. |
| 2026-07-13 19:58:05 +0000 | <bwe> | [exa]: we brought parserFactory down to 4% of total time consumed (newest profiling). |
| 2026-07-13 20:01:28 +0000 | <bwe> | btw what's the OldList stuff in base. Will it eventually be deprecated? |
| 2026-07-13 20:06:26 +0000 | <davean> | bwe: I think the documentation it contains explains pretty clearly - what are you left wondering? |
| 2026-07-13 20:10:08 +0000 | <bwe> | davean: you mean this? https://hackage-content.haskell.org/package/ghc-internal-9.1401.0/docs/GHC-Internal-Data-OldList.h… -- it doesn't clarify whether it'll be deprecated or not. Neither does https://github.com/haskell/core-libraries-committee/blob/main/guides/warning-for-head-and-tail.md -- what do I miss? |
| 2026-07-13 20:11:34 +0000 | <davean> | No I don't mean that, I mean https://hackage-content.haskell.org/package/base-4.22.0.0/docs/GHC-OldList.html |
| 2026-07-13 20:11:45 +0000 | <davean> | This has nothing to do with head and tail |
| 2026-07-13 20:11:55 +0000 | <davean> | You asked about base, then linked things not in base. |
| 2026-07-13 20:13:11 +0000 | target_i | (~target_i@user/target-i/x-6023099) (Quit: leaving) |
| 2026-07-13 20:13:12 +0000 | <davean> | It *started life deprecated* |
| 2026-07-13 20:14:13 +0000 | <bwe> | davean: ok, now, it's clear. Thanks. I ended up on the wrong OldList with hoogle. |
| 2026-07-13 20:25:13 +0000 | <davean> | I bet it'll be around for 30 years because no one is motivated to do even the minor updates to remove it and having it is close to cost free, but it never was there to be supported or last. Its a transition mechanism. |
| 2026-07-13 20:30:12 +0000 | lisbeths | (uid135845@id-135845.lymington.irccloud.com) lisbeths |
| 2026-07-13 20:34:23 +0000 | sim590_ | (~sim590@209-15-184-6.resi.cgocable.ca) (Read error: Connection reset by peer) |
| 2026-07-13 20:34:37 +0000 | sim590 | (~sim590@2001:18c0:a82:2400::9fb) sim590 |
| 2026-07-13 20:56:40 +0000 | ksqsf | (~ksqsf@2a02:810a:8d31:1600:2cd7:fe24:5e46:7daa) ksqsf |
| 2026-07-13 20:57:33 +0000 | Pozyomka_ | Pozyomka |
| 2026-07-13 21:03:46 +0000 | Googulator | (~Googulato@193-226-241-68.pool.digikabel.hu) (Quit: Client closed) |
| 2026-07-13 21:04:08 +0000 | Googulator | (~Googulato@193-226-241-68.pool.digikabel.hu) |
| 2026-07-13 21:10:01 +0000 | sim590 | (~sim590@2001:18c0:a82:2400::9fb) (Quit: ZNC 1.10.2 - https://znc.in) |
| 2026-07-13 21:12:25 +0000 | Lord_of_Life | (~Lord@user/lord-of-life/x-2819915) (Remote host closed the connection) |
| 2026-07-13 21:16:16 +0000 | Lord_of_Life | (~Lord@user/lord-of-life/x-2819915) Lord_of_Life |
| 2026-07-13 21:17:12 +0000 | emilym | (~Thunderbi@user/emilym) emilym |
| 2026-07-13 21:19:36 +0000 | ksqsf | (~ksqsf@2a02:810a:8d31:1600:2cd7:fe24:5e46:7daa) (Quit: clatter.el) |
| 2026-07-13 21:19:48 +0000 | sim590 | (~sim590@2001:18c0:a82:2400::9fb) sim590 |
| 2026-07-13 21:20:05 +0000 | ksqsf | (~user@2a02:810a:8d31:1600:2cd7:fe24:5e46:7daa) ksqsf |
| 2026-07-13 21:21:09 +0000 | Lord_of_Life | (~Lord@user/lord-of-life/x-2819915) (Ping timeout: 245 seconds) |
| 2026-07-13 21:21:34 +0000 | emilym | (~Thunderbi@user/emilym) (Ping timeout: 245 seconds) |
| 2026-07-13 21:24:59 +0000 | <monochrom> | This is why I don't use google or hoogle. |
| 2026-07-13 21:25:36 +0000 | <monochrom> | In particular google always returns the version you aren't using. |
| 2026-07-13 21:26:53 +0000 | <davean> | Yah I don't get why you would ask a question about something specific and not just ... go directly there. |
| 2026-07-13 21:27:21 +0000 | <davean> | Like anything else is defiantely the wrong place to take advice from. The only reason to use such a tool is if it gets you to exactly that one place you know you're supposed to go faster |