Newest at the top
| 2025-07-28 17:45:03 +0200 | trickard_ | trickard |
| 2025-07-28 17:43:31 +0200 | trickard_ | (~trickard@cpe-51-98-47-163.wireline.com.au) |
| 2025-07-28 17:43:18 +0200 | trickard_ | (~trickard@cpe-51-98-47-163.wireline.com.au) (Read error: Connection reset by peer) |
| 2025-07-28 17:41:37 +0200 | Unicorn_Princess | (~Unicorn_P@user/Unicorn-Princess/x-3540542) Unicorn_Princess |
| 2025-07-28 17:37:41 +0200 | AVA | (~AVA@84.54.80.216) (Read error: Connection reset by peer) |
| 2025-07-28 17:28:31 +0200 | <Henson> | merijn: yes |
| 2025-07-28 17:28:13 +0200 | <merijn> | ok, so you do specifically need malloc :) |
| 2025-07-28 17:28:05 +0200 | gmg | (~user@user/gehmehgeh) gehmehgeh |
| 2025-07-28 17:27:48 +0200 | <Henson> | merijn: the problem with that is that in my production code I don't know a-priori the amount of memory that should be allocated, because it's being allocated in the C/C++ layer and passed back to Haskell. |
| 2025-07-28 17:26:43 +0200 | <merijn> | https://hackage.haskell.org/package/base-4.21.0.0/docs/Foreign-ForeignPtr.html#v:mallocForeignPtrB… |
| 2025-07-28 17:26:40 +0200 | <merijn> | I've always used Foreign.ForeignPtr, so I was guessing that's why I'm used to seeing it in profiles while you might not |
| 2025-07-28 17:25:51 +0200 | <merijn> | Since it uses GHC's newAlignedPinnedByteArray, rather than malloc |
| 2025-07-28 17:25:16 +0200 | <merijn> | Henson: I meant more like, using mallocForeignPtrBytes, which allocates using GHC's allocator and SHOULD appear in memory profiles |
| 2025-07-28 17:24:41 +0200 | <Henson> | merijn: the problem also happens with C++ "new" and "delete" |
| 2025-07-28 17:24:13 +0200 | trickard_ | (~trickard@cpe-51-98-47-163.wireline.com.au) |
| 2025-07-28 17:24:11 +0200 | <merijn> | Henson: Do you specifically need C malloc or just any allocation? |
| 2025-07-28 17:23:58 +0200 | <Henson> | merijn: oh, also, the leak when using "async" will not happen with "+RTS -N1" |
| 2025-07-28 17:23:43 +0200 | trickard | (~trickard@cpe-51-98-47-163.wireline.com.au) (Ping timeout: 245 seconds) |
| 2025-07-28 17:23:34 +0200 | <merijn> | (not your leak problem, the profiling one) |
| 2025-07-28 17:23:23 +0200 | <merijn> | The problem is calloc :) |
| 2025-07-28 17:23:17 +0200 | <merijn> | Henson: Ah, wait |
| 2025-07-28 17:22:40 +0200 | <Henson> | merijn: I don't think it does. I've done every kind of memory profiling the GHC's profiling offers, and the C-based memory allocation doesn't show up. |
| 2025-07-28 17:22:40 +0200 | sam113102 | sam113101 |
| 2025-07-28 17:22:36 +0200 | sam113101 | (~sam@modemcable200.189-202-24.mc.videotron.ca) (Read error: Connection reset by peer) |
| 2025-07-28 17:22:33 +0200 | sam113102 | (~sam@modemcable200.189-202-24.mc.videotron.ca) sam113101 |
| 2025-07-28 17:21:03 +0200 | <merijn> | I would expect that to still show up as a PINNED allocation |
| 2025-07-28 17:21:03 +0200 | <Henson> | merijn, magic_rb: I'm working on putting together a minimal reproducer. |
| 2025-07-28 17:20:15 +0200 | <Henson> | merijn: I have tried profiling, but the problem is that the calloc returns a pointer, which is tiny, but points to a large amount of memory. Because the memory is allocated in the C level, the RTS isn't able to keep track of it, so no significant memory usage shows up. |
| 2025-07-28 17:19:21 +0200 | <merijn> | That said, this is hard to troubleshoot without a minimal reproducer like magic_rb said |
| 2025-07-28 17:18:31 +0200 | <merijn> | Or checked the GC report? |
| 2025-07-28 17:18:00 +0200 | <merijn> | Henson: Have you profiled yet? |
| 2025-07-28 17:17:30 +0200 | <Henson> | merijn: yes, just to "cancel" them at the end. If I don't keep the async values then the problem still occurs. The memory usage also seems to follow a pattern of flat usage for several seconds, then an increase in about 2 seconds to a higher value, then it flattens for maybe 7 seconds, then another gradual increase, etc. |
| 2025-07-28 17:16:08 +0200 | <merijn> | Not to mention sandboxes involved tedious manual management |
| 2025-07-28 17:15:39 +0200 | fp | (~Thunderbi@2001:708:20:1406::10c5) (Ping timeout: 260 seconds) |
| 2025-07-28 17:14:29 +0200 | <dolio> | In the old sandbox setup, each (transitive) dependency would only ever be built once per sandbox . But there was no sharing of common dependencies between sandboxes, even if they wanted identical build specifications. |
| 2025-07-28 17:13:23 +0200 | <merijn> | Henson: The question is: Are you keeping Async values around somewhere? |
| 2025-07-28 17:10:45 +0200 | lortabac | (~lortabac@2a01:e0a:541:b8f0:55ab:e185:7f81:54a4) (Quit: WeeChat 4.5.2) |
| 2025-07-28 17:09:35 +0200 | <dolio> | And the analogy with cabal is that common dependencies of multiple projects are built at most once and shared, rather than being built once for each project. |
| 2025-07-28 17:09:30 +0200 | kuribas | (~user@ptr-17d51epjby9myrcgbo7.18120a2.ip6.access.telenet.be) (Ping timeout: 244 seconds) |
| 2025-07-28 17:09:05 +0200 | <Henson> | the leak seems to happen equally with GHC 8.10.7 and 9.12.2 |
| 2025-07-28 17:08:28 +0200 | <Henson> | merijn: I also tried using forkIO and forkOS instead of async, and the problem still happens. |
| 2025-07-28 17:08:10 +0200 | <Henson> | merijn: the creator/destructor threads are created in pairs and use an MVar to pass the memory pointers back-and-forth. Only 10 creator/destructor pairs are ever created, but many many memory allocations/deallocations are performed. When the memory block allocated is a multiple of 1024*1024 (1 MB) then the leak also doesn't happen. Based on the block size the leak can be faster or slow |
| 2025-07-28 17:07:42 +0200 | <dolio> | CSE is 'common subexpression elimination.' It's a transformation where you look for subexpressions that occur multiple times, then turn them into named bindings so that they are evaluated at most once. |
| 2025-07-28 17:03:29 +0200 | remmie | (ianremsen@tilde.team) remsense |
| 2025-07-28 17:00:46 +0200 | trickard_ | trickard |
| 2025-07-28 16:55:58 +0200 | vulpine | (xfnw@user/meow/xfnw) xfnw |
| 2025-07-28 16:55:53 +0200 | LainIwakura | (~LainIwaku@user/LainIwakura) (Ping timeout: 272 seconds) |
| 2025-07-28 16:48:25 +0200 | AVA | (~AVA@84.54.80.216) |
| 2025-07-28 16:40:31 +0200 | Lycurgus | (~juan@user/Lycurgus) (Quit: irc.renjuan.org (juan@acm.org)) |
| 2025-07-28 16:38:08 +0200 | <merijn> | So if you're holding on to async's the threads don't get GCed and (in turn) nothing in that thread's GC root will be |