Newest at the top
| 2025-02-25 10:17:02 +0100 | <Athas> | This looks compelling. Using 'f (VS.Vector a)' instead of just 'a' is make it easier to have some efficient representation without imposing constraints on 'f'? |
| 2025-02-25 10:15:48 +0100 | <tomsmeding> | also that |
| 2025-02-25 10:15:40 +0100 | <Athas> | Yes, that can be fixed at a higher level. |
| 2025-02-25 10:15:12 +0100 | <tomsmeding> | but that's rather non-fundamental |
| 2025-02-25 10:15:04 +0100 | <tomsmeding> | arrays are single-dimensional only |
| 2025-02-25 10:14:57 +0100 | <tomsmeding> | full disclosure: I've been hacking on something like this; the API currently looks like this: gradient' from here ( https://git.tomsmeding.com/ad-dual/tree/src/Numeric/ADDual/Array/Internal.hs ), where 'VDual s Double' implements the classes from here ( https://git.tomsmeding.com/ad-dual/tree/src/Numeric/ADDual/VectorOps.hs ) |
| 2025-02-25 10:13:43 +0100 | <Athas> | When you're talking about the Jacobian matrix, you're already in a sense flattening all of the scalars to form each axis. |
| 2025-02-25 10:13:01 +0100 | <tomsmeding> | I'm not thinking about functional vector fields :p |
| 2025-02-25 10:12:52 +0100 | AlexZenon | (~alzenon@178.34.162.44) (Ping timeout: 272 seconds) |
| 2025-02-25 10:12:43 +0100 | <Athas> | I'd much prefer a Haskell AD library that can efficiently handle boring classic numerical problems, even if fancy weird things, like functional vector fields, cannot be handled. |
| 2025-02-25 10:12:20 +0100 | <Athas> | These libraries ultimately expect the input to be a sequence of scalars. In Haskell, you could do fancier things (like arrays of functions!), but I'm not sure that is so useful. |
| 2025-02-25 10:12:01 +0100 | <tomsmeding> | well f (Vector Double) would be rather easy to support, for a Traversable f |
| 2025-02-25 10:11:35 +0100 | <Athas> | You just pretend the arrays are concatenated. It's easy. |
| 2025-02-25 10:11:28 +0100 | <tomsmeding> | interesting |
| 2025-02-25 10:11:22 +0100 | <Athas> | It's common in C++ AD libraries yes, although they sometimes put some polish on top to make it seem like that's not how it works. |
| 2025-02-25 10:11:18 +0100 | <tomsmeding> | which works for scalars, but if you have array support, then it's suddenly inadequate |
| 2025-02-25 10:11:05 +0100 | <tomsmeding> | exactly |
| 2025-02-25 10:10:58 +0100 | <Athas> | But I don't understand - if you require that the input is some 'f t' where 'f' is Traversable, isn't that essentially going to be a sequence of 't's with some shape on top? |
| 2025-02-25 10:10:57 +0100 | <tomsmeding> | that feels a bit much, to me |
| 2025-02-25 10:10:50 +0100 | <tomsmeding> | do other AD libraries really do that? |
| 2025-02-25 10:10:40 +0100 | <Athas> | Not the list, just 'Vector a'. |
| 2025-02-25 10:10:23 +0100 | <tomsmeding> | what do you mean with a monomorphic sequence of scalars? [Vector a]? |
| 2025-02-25 10:09:55 +0100 | <Athas> | Well, so there's another restriction I'm OK with: the input may be assumed to be isomorphic to a monomorphic sequence of scalars. Many AD libraries make that assumption. The Haskell type system is easily strong enough to make that nice to work with. |
| 2025-02-25 10:09:34 +0100 | <tomsmeding> | I was thinking of that Traversable constraint; that requires all the "input values" to really be exactly the same Haskell type |
| 2025-02-25 10:09:14 +0100 | <Athas> | Oh, I was considering only the types of the underlying scalars. |
| 2025-02-25 10:08:46 +0100 | <tomsmeding> | as is "arrays and scalars" |
| 2025-02-25 10:08:29 +0100 | <tomsmeding> | Athas: mind that having arrays of different ranks in the input is already "heterogeneously typed" |
| 2025-02-25 10:08:15 +0100 | <tomsmeding> | Athas: I tried, but the fact that the forward pass becomes a write to a vector with potential reallocation logic, instead of just allocating an object in the haskell nursery, increases the overhead quite a bit |
| 2025-02-25 10:08:09 +0100 | <Athas> | For example, they occur in none of the ADBench benchmarks. |
| 2025-02-25 10:07:50 +0100 | <Athas> | Heterogeneously typed active inputs are rare. |
| 2025-02-25 10:07:30 +0100 | <Athas> | The tape-based AD libraries in C++ are not slow at all. |
| 2025-02-25 10:07:19 +0100 | <tomsmeding> | Athas: some other component of the interface that is surprisingly tricky: 'ad' uses Traversable as the universal input structure, but if you have heterogeneously typed inputs, that doesn't really work any more |
| 2025-02-25 10:07:16 +0100 | <Athas> | Making the tape more efficient (by storing it in an unboxed and bump-allocated vector) would make a big difference to performance, I think. |
| 2025-02-25 10:06:57 +0100 | <Athas> | I would be OK with the API limiting primal values to types that can be unboxed (and also imposing some strictness requirements). |
| 2025-02-25 10:06:21 +0100 | <tomsmeding> | (though perhaps allocating fewer list nodes is doable) |
| 2025-02-25 10:05:57 +0100 | <tomsmeding> | as in, making them work but slow is easy (fromList . map f . toList), but making them fast is rather difficult |
| 2025-02-25 10:05:47 +0100 | <Athas> | Yes, I know, which is why I wouldn't expect them to work well, and I could sympathise with not having them at all. |
| 2025-02-25 10:05:28 +0100 | <tomsmeding> | Athas: Thanks! Second order operations are fundamentally more difficult to make work with the 'ad'-style algorithm than first-order bulk operations, though |
| 2025-02-25 10:03:21 +0100 | why | (~why@n218250229238.netvigator.com) |
| 2025-02-25 10:02:09 +0100 | weary-traveler | (~user@user/user363627) (Remote host closed the connection) |
| 2025-02-25 10:01:22 +0100 | <Athas> | Then the array type has a vocabulary of first order bulk operations (where I expect good performance for derivatives), and also some second order operations (like map) that should at least work. |
| 2025-02-25 10:01:07 +0100 | chele | (~chele@user/chele) chele |
| 2025-02-25 10:00:44 +0100 | <Athas> | tomsmeding: good question. I'm not terribly imaginatine when it comes to Haskell types, I must admit. I would probably like a type family 'Array sh t', where 'sh' is an Accelerate-like shape (this is not so important for 'ad'), and 't' is heavily constrained to either a some primal number type or dual/adjoint numbers. |
| 2025-02-25 10:00:01 +0100 | acidjnk | (~acidjnk@p200300d6e7283f38a15cd1ba33b15ba0.dip0.t-ipconnect.de) acidjnk |
| 2025-02-25 09:58:45 +0100 | tabaqui1 | (~root@87.200.129.102) tabaqui |
| 2025-02-25 09:58:41 +0100 | zungi | (~tory@user/andrewchawk) andrewchawk |
| 2025-02-25 09:57:50 +0100 | lxsameer | (~lxsameer@Serene/lxsameer) lxsameer |
| 2025-02-25 09:57:12 +0100 | zungi | (~tory@user/andrewchawk) (Ping timeout: 264 seconds) |
| 2025-02-25 09:52:36 +0100 | alfiee | (~alfiee@user/alfiee) (Ping timeout: 272 seconds) |
| 2025-02-25 09:49:22 +0100 | <tomsmeding> | Athas: suppose I had a library inspired by Kmett's 'ad' library, but with first-class support for arrays somehow, instead of just scalars. (The goal being to significantly reduce the overhead of all those dual-number heap nodes.) What would you expect its interface to be? |