Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions conformance/text.json
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
{ "name": "one BOM is removed from the start of every record", "input": "\ufeffa\n\ufeffb", "options": { "trim": false }, "expect": ["a", "b"] },
{ "name": "only one BOM per record", "input": "\ufeff\ufeffa", "options": { "trim": false }, "expect": ["\ufeffa"] },
{ "name": "trim uses the ECMAScript whitespace set", "input": "\u00a0a\u3000\n\u2028b\ufeff\n\u205fc\u1680", "expect": ["a", "b", "c"] },
{ "name": "a row of only Unicode whitespace is blank", "input": "a\n\u00a0\u3000\ufeff\nb", "expect": ["a", "b"] },
{ "name": "NEL is not whitespace to trim", "input": "\u0085a\u0085", "expect": ["\u0085a\u0085"] },
{ "name": "information separators are not whitespace to trim", "input": "\u001ca\u001f", "expect": ["\u001ca\u001f"] },
{ "name": "invalid UTF-8 decodes lossily, one replacement per maximal subpart", "input": { "base64": "Yf9iCuKC" }, "expect": ["a\ufffdb", "\ufffd"] },
Expand Down
12 changes: 12 additions & 0 deletions crates/spliterator-py/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,18 @@ for line in spliterator.read_text(b" a \n\n b\n"): # "a", "b"
...
```

## Async and counts

Each reader has an async twin that reads each batch on a worker thread, so the event loop never blocks on I/O:

```python
async for row in spliterator.aread_csv("people.csv"):
...
```

`count_text`, `count_csv`, `count_tsv` and `count_psv` return the number of rows the matching reader would yield,
without building them.

## Sources

A source is a path (`str` or `os.PathLike`), bytes-like data, or a binary file object.
Expand Down
Loading
Loading