Newest at the top
2025-03-26 12:08:09 +0100 | <tomsmeding> | I can imagine that it doesn't matter much in practice |
2025-03-26 12:07:51 +0100 | <Athas> | There are, actually, in the runtime system! |
2025-03-26 12:07:45 +0100 | <Athas> | Yes, but that doesn't really make much of a difference. |
2025-03-26 12:07:42 +0100 | <tomsmeding> | there are no such stride vectors at runtime any more in Futhark, I presume |
2025-03-26 12:07:25 +0100 | <tomsmeding> | but you do it statically, right? |
2025-03-26 12:07:00 +0100 | <Athas> | Yeah, it's an LMAD. We use the exact same representation in Futhark. |
2025-03-26 12:06:50 +0100 | <tomsmeding> | this probably has a name |
2025-03-26 12:06:37 +0100 | <tomsmeding> | it's this: https://hackage.haskell.org/package/orthotope-0.1.7.0/docs/Data-Array-Internal.html#t:T |
2025-03-26 12:06:13 +0100 | <Athas> | What is that representation? Is it documented anywhere? |
2025-03-26 12:06:09 +0100 | alfiee | (~alfiee@user/alfiee) (Ping timeout: 268 seconds) |
2025-03-26 12:05:55 +0100 | <tomsmeding> | and consumers of arrays can inspect what the actual strides are, to optimise their operation |
2025-03-26 12:05:35 +0100 | <tomsmeding> | replicate and transpose are O(number of dimensions) |
2025-03-26 12:05:20 +0100 | <tomsmeding> | it doesn't, the orthotope strided array representation does |
2025-03-26 12:05:08 +0100 | <Athas> | How does fast arithmetic imply fusion? |
2025-03-26 12:04:54 +0100 | merijn | (~merijn@77.242.116.146) (Ping timeout: 260 seconds) |
2025-03-26 12:04:46 +0100 | <tomsmeding> | so you get some form of simple runtime fusion |
2025-03-26 12:04:37 +0100 | <tomsmeding> | it's orthotope (on hackage), but with arithmetic operations that are actually fast |
2025-03-26 12:04:19 +0100 | infinity0 | (~infinity0@pwned.gg) (Ping timeout: 260 seconds) |
2025-03-26 12:04:15 +0100 | <tomsmeding> | Athas: there's also a fun array library underneath horde-ad, actually, that I wrote as a big yak shave: https://git.tomsmeding.com/ox-arrays |
2025-03-26 12:03:13 +0100 | <tomsmeding> | shelf? |
2025-03-26 12:03:11 +0100 | <tomsmeding> | there is probably a way to write this up properly so that it's publishable, but among us, we've spent enough hours on trying to write it down that we're likely to just shelve it. |
2025-03-26 12:02:51 +0100 | <Athas> | Mikolaj is doing almost ten commits per day, so I can't imagine it will be long before it works! |
2025-03-26 12:02:25 +0100 | <tomsmeding> | if it worked out just a little bit more, it would work just fine, but as it is, it's complex _and_ has annoying limitations |
2025-03-26 12:02:04 +0100 | fp | (~Thunderbi@2001:708:20:1406::1370) fp |
2025-03-26 12:01:51 +0100 | <tomsmeding> | horde-ad tries to do quite a bit more than ad-dual, though, and there the tricky bit is that there are lots of parts where the algorithm has unfortunate limitations |
2025-03-26 12:01:19 +0100 | alfiee | (~alfiee@user/alfiee) alfiee |
2025-03-26 12:01:11 +0100 | <tomsmeding> | I'm not sure that's a paper. |
2025-03-26 12:01:04 +0100 | <tomsmeding> | number goes down as in what my ad-dual hack does is compelling as in: it's much faster than something that's very crap compared to the state of the art |
2025-03-26 12:00:41 +0100 | xff0x | (~xff0x@2405:6580:b080:900:6185:acb4:363b:5fd3) |
2025-03-26 12:00:32 +0100 | <Athas> | "Number goes down" is not compelling? Or is it the actual explanation that is difficult to get right? |
2025-03-26 12:00:30 +0100 | <tomsmeding> | the possibility exists that it will just be a library and a "here's some text" on arXiv |
2025-03-26 12:00:01 +0100 | <tomsmeding> | we're tried to write it up, and we have some kind of write-up, in fact, but we've been struggling to find a presentation that is compelling enough for a successful publication |
2025-03-26 11:58:57 +0100 | <tomsmeding> | the point of horde-ad is that it's an extension to that |
2025-03-26 11:58:51 +0100 | <tomsmeding> | yes, indeed |
2025-03-26 11:58:49 +0100 | <tomsmeding> | oh, it is, at some point -- but that paper essentially just describes 'ad' |
2025-03-26 11:58:45 +0100 | <Athas> | But then that paper says that it's really just a different exposition of how 'ad' works, so who knows. |
2025-03-26 11:58:33 +0100 | <Athas> | The horde-ad README says it's based on "Provably Correct, Asymptotically Efficient, Higher-Order Reverse-Mode Automatic Differentiation"! |
2025-03-26 11:58:08 +0100 | <tomsmeding> | horde-ad is based on unpublished stuff :P |
2025-03-26 11:57:55 +0100 | <tomsmeding> | Athas: what paper? |
2025-03-26 11:57:42 +0100 | merijn | (~merijn@77.242.116.146) merijn |
2025-03-26 11:57:14 +0100 | <Athas> | tomsmeding: so when will horde-ad be ready? I have (re?)-read the paper and now I am excited. |
2025-03-26 11:57:03 +0100 | merijn | (~merijn@77.242.116.146) (Ping timeout: 246 seconds) |
2025-03-26 11:56:21 +0100 | dhil | (~dhil@2a0c:b381:52e:3600:79cd:9b5b:3ea2:beda) dhil |
2025-03-26 11:49:53 +0100 | hattckory | (~hattckory@bras-base-toroon4524w-grc-30-70-27-118-207.dsl.bell.ca) (Ping timeout: 248 seconds) |
2025-03-26 11:48:49 +0100 | coldtom8 | (~coldtom@coldrick.cc) () |
2025-03-26 11:48:15 +0100 | coldtom8 | (~coldtom@coldrick.cc) coldtom |
2025-03-26 11:46:20 +0100 | ash3en | (~Thunderbi@149.222.157.169) (Remote host closed the connection) |
2025-03-26 11:45:40 +0100 | hattckory | (~hattckory@bras-base-toroon4524w-grc-30-70-27-118-207.dsl.bell.ca) |
2025-03-26 11:42:22 +0100 | ash3en | (~Thunderbi@149.222.157.169) ash3en |
2025-03-26 11:39:44 +0100 | ash3en | (~Thunderbi@149.222.157.169) (Quit: ash3en) |