Newest at the top
| 2025-03-13 15:26:11 +0100 | tabaqui | (~tabaqui@167.71.80.236) tabaqui |
| 2025-03-13 15:25:41 +0100 | tabaqui | (~tabaqui@167.71.80.236) (Quit: WeeChat 4.5.1) |
| 2025-03-13 15:17:59 +0100 | <kuribas> | oh, there is .npy format... |
| 2025-03-13 15:12:03 +0100 | alfiee | (~alfiee@user/alfiee) (Ping timeout: 245 seconds) |
| 2025-03-13 15:07:47 +0100 | alfiee | (~alfiee@user/alfiee) alfiee |
| 2025-03-13 14:58:12 +0100 | <kuribas> | I wanted to use parquet, but nothing exists for haskell. |
| 2025-03-13 14:57:19 +0100 | <kuribas> | So the quickest thing would be to use my haskell version. |
| 2025-03-13 14:57:10 +0100 | <kuribas> | But now I see it comes with a noncommercial license. |
| 2025-03-13 14:56:59 +0100 | <kuribas> | The thing is, I implemented some compression code in haskell that we used at work. Now I need a python version, so I thought to just use the C library and bind it. |
| 2025-03-13 14:55:07 +0100 | <kuribas> | it's just a timeseries. |
| 2025-03-13 14:55:00 +0100 | <kuribas> | comerijn: 1 |
| 2025-03-13 14:53:54 +0100 | <comerijn> | Because if the answers is >1 then you need to worry about row vs column major for numpy |
| 2025-03-13 14:53:12 +0100 | <comerijn> | kuribas: What dimensionality for the array? |
| 2025-03-13 14:52:59 +0100 | <comerijn> | Evil voice: Just dump out a Storabel Vector ;) |
| 2025-03-13 14:52:06 +0100 | <kuribas> | right |
| 2025-03-13 14:51:54 +0100 | <dminuoso> | Unstructured, but perhaps suitable |
| 2025-03-13 14:51:46 +0100 | <dminuoso> | Which is a very dense and high performance encoding |
| 2025-03-13 14:51:33 +0100 | <dminuoso> | Python library exists too. |
| 2025-03-13 14:51:28 +0100 | <dminuoso> | https://hackage.haskell.org/package/msgpack |
| 2025-03-13 14:51:24 +0100 | <kuribas> | oh, numpy.frombuffer, then I just need to emit the right format. |
| 2025-03-13 14:51:12 +0100 | <dminuoso> | kuribas: You could also just use msgpack. |
| 2025-03-13 14:50:40 +0100 | <kuribas> | I'd like to read it into python as a numpy array. |
| 2025-03-13 14:50:24 +0100 | <dminuoso> | Indeed. |
| 2025-03-13 14:50:11 +0100 | <kuribas> | I don't even need CSV. |
| 2025-03-13 14:50:00 +0100 | <kuribas> | Right, I could just use "binary" to emit a blob... |
| 2025-03-13 14:49:45 +0100 | sabathan | (~sabathan@amarseille-159-1-12-107.w86-203.abo.wanadoo.fr) |
| 2025-03-13 14:49:27 +0100 | <dminuoso> | (Only compression will make it more compact) |
| 2025-03-13 14:49:10 +0100 | <dminuoso> | The denses way to pack IEE754 is to just dump the representation. |
| 2025-03-13 14:49:06 +0100 | <kuribas> | I could also base64 encode a binary blob. |
| 2025-03-13 14:48:48 +0100 | <dminuoso> | You just need to ensure its properly quoted, then it does not even matter whats inside the quotes as long as the other side can parse it. |
| 2025-03-13 14:48:43 +0100 | <kuribas> | I mean that CSV is not dense by default... |
| 2025-03-13 14:48:18 +0100 | <dminuoso> | kuribas: What do you mean by "is that a thing"? |
| 2025-03-13 14:47:57 +0100 | <kuribas> | or timestmaps and integers |
| 2025-03-13 14:47:52 +0100 | <kuribas> | It's timestamp and floats |
| 2025-03-13 14:47:44 +0100 | <dminuoso> | i.e. if its floats, just emit their IEEE754 representation |
| 2025-03-13 14:47:43 +0100 | <kuribas> | Is that a thing? |
| 2025-03-13 14:47:14 +0100 | <dminuoso> | Why not CSV, but with dense representation? |
| 2025-03-13 14:46:01 +0100 | sabathan | (~sabathan@amarseille-159-1-12-107.w86-203.abo.wanadoo.fr) (Read error: Connection reset by peer) |
| 2025-03-13 14:42:55 +0100 | <kuribas> | I could export csv, but it'd be nicer to have something compact. |
| 2025-03-13 14:42:40 +0100 | <kuribas> | Is there a library to export numerical data that can be read in python, preferably compressed? |
| 2025-03-13 14:39:40 +0100 | infohazards | (~user@user/fmira) fmira |
| 2025-03-13 14:39:13 +0100 | infohazards | (~user@user/fmira) (Remote host closed the connection) |
| 2025-03-13 14:33:44 +0100 | fp | (~Thunderbi@wireless-86-50-141-158.open.aalto.fi) (Ping timeout: 260 seconds) |
| 2025-03-13 14:30:39 +0100 | xff0x | (~xff0x@ai066236.d.east.v6connect.net) |
| 2025-03-13 14:30:14 +0100 | hsw | (~hsw@112-104-12-126.adsl.dynamic.seed.net.tw) hsw |
| 2025-03-13 14:28:59 +0100 | jespada | (~jespada@r179-25-57-145.dialup.adsl.anteldata.net.uy) jespada |
| 2025-03-13 14:28:45 +0100 | sprotte24 | (~sprotte24@p200300d16f0b61000c042d391325af51.dip0.t-ipconnect.de) (Read error: Connection reset by peer) |
| 2025-03-13 14:27:45 +0100 | xff0x | (~xff0x@2405:6580:b080:900:2020:1277:8563:d3ac) (Ping timeout: 248 seconds) |
| 2025-03-13 14:26:30 +0100 | alfiee | (~alfiee@user/alfiee) (Ping timeout: 252 seconds) |
| 2025-03-13 14:25:22 +0100 | infohazards | (~user@user/fmira) fmira |