|
| 1 | +# SimdXml |
| 2 | + |
| 3 | +SIMD-accelerated XML parsing with full XPath 1.0 support for Elixir. |
| 4 | + |
| 5 | +SimdXml parses XML into a flat structural index (~16 bytes per tag) using SIMD |
| 6 | +instructions, then evaluates XPath expressions against it using array operations. |
| 7 | +No DOM tree, no atom creation from untrusted input, no XXE vulnerabilities. |
| 8 | + |
| 9 | +Wraps the [simdxml](https://crates.io/crates/simdxml) Rust crate via |
| 10 | +[Rustler](https://github.com/rusterlium/rustler) NIFs with precompiled binaries |
| 11 | +for all major platforms. |
| 12 | + |
| 13 | +## Installation |
| 14 | + |
| 15 | +```elixir |
| 16 | +def deps do |
| 17 | + [{:simdxml, "~> 0.1.0"}] |
| 18 | +end |
| 19 | +``` |
| 20 | + |
| 21 | +Precompiled NIF binaries are provided for macOS (Apple Silicon, Intel), Linux |
| 22 | +(x86_64, aarch64, musl), and Windows. Set `SIMDXML_BUILD=1` to compile from |
| 23 | +source if needed. |
| 24 | + |
| 25 | +## Quick start |
| 26 | + |
| 27 | +```elixir |
| 28 | +# Parse |
| 29 | +doc = SimdXml.parse!("<library><book lang='en'><title>Elixir</title></book></library>") |
| 30 | + |
| 31 | +# Query with XPath |
| 32 | +SimdXml.xpath_text!(doc, "//title") |
| 33 | +#=> ["Elixir"] |
| 34 | + |
| 35 | +# Navigate elements (Enumerable) |
| 36 | +root = SimdXml.Document.root(doc) |
| 37 | +Enum.map(root, & &1.tag) |
| 38 | +#=> ["book"] |
| 39 | + |
| 40 | +# Attributes |
| 41 | +[book] = SimdXml.Element.children(root) |
| 42 | +SimdXml.Element.get(book, "lang") |
| 43 | +#=> "en" |
| 44 | +``` |
| 45 | + |
| 46 | +## Query combinators |
| 47 | + |
| 48 | +Build XPath queries with Elixir pipes instead of strings: |
| 49 | + |
| 50 | +```elixir |
| 51 | +import SimdXml.Query |
| 52 | + |
| 53 | +query = descendant("book") |> where_attr("lang", "en") |> child("title") |> text() |
| 54 | + |
| 55 | +SimdXml.query!(doc, query) |
| 56 | +#=> ["Elixir"] |
| 57 | + |
| 58 | +# Inspect the generated XPath |
| 59 | +SimdXml.Query.to_xpath(query) |
| 60 | +#=> "//book[@lang='en']/title/text()" |
| 61 | +``` |
| 62 | + |
| 63 | +Queries are composable data structures — extract common fragments and reuse them: |
| 64 | + |
| 65 | +```elixir |
| 66 | +books = descendant("book") |
| 67 | +english = books |> where_attr("lang", "en") |
| 68 | +titles = english |> child("title") |> text() |
| 69 | +authors = english |> child("author") |> text() |
| 70 | +``` |
| 71 | + |
| 72 | +## Compiled queries |
| 73 | + |
| 74 | +Compile once, evaluate against many documents: |
| 75 | + |
| 76 | +```elixir |
| 77 | +query = SimdXml.compile!("//title") |
| 78 | + |
| 79 | +SimdXml.eval_text!(doc1, query) |
| 80 | +SimdXml.eval_text!(doc2, query) |
| 81 | + |
| 82 | +# Optimized short-circuit operations |
| 83 | +SimdXml.eval_count!(doc, query) #=> 1 |
| 84 | +SimdXml.eval_exists?(doc, query) #=> {:ok, true} |
| 85 | +``` |
| 86 | + |
| 87 | +Compiled queries are NIF resources — safe to share across processes, store in |
| 88 | +ETS, or hold in module attributes. |
| 89 | + |
| 90 | +## Batch processing |
| 91 | + |
| 92 | +Process thousands of documents with bloom filter prescanning: |
| 93 | + |
| 94 | +```elixir |
| 95 | +query = SimdXml.compile!("//claim") |
| 96 | +{:ok, results} = SimdXml.Batch.eval_text_bloom(xml_binaries, query) |
| 97 | +``` |
| 98 | + |
| 99 | +Documents that cannot contain the target tags are skipped without parsing. |
| 100 | + |
| 101 | +## Quick grep mode |
| 102 | + |
| 103 | +For simple `//tagname` extraction at memory bandwidth — no structural index: |
| 104 | + |
| 105 | +```elixir |
| 106 | +scanner = SimdXml.Quick.new("claim") |
| 107 | +SimdXml.Quick.extract_first(scanner, xml) #=> "First claim text" |
| 108 | +SimdXml.Quick.exists?(scanner, xml) #=> true |
| 109 | +SimdXml.Quick.count(scanner, xml) #=> 42 |
| 110 | +``` |
| 111 | + |
| 112 | +## Result helpers |
| 113 | + |
| 114 | +```elixir |
| 115 | +SimdXml.Result.one(doc, "//title") #=> "Elixir" |
| 116 | +SimdXml.Result.fetch(doc, "//title") #=> {:ok, "Elixir"} |
| 117 | +SimdXml.Result.all(doc, "//title") #=> ["Elixir"] |
| 118 | +``` |
| 119 | + |
| 120 | +## Why SimdXml? |
| 121 | + |
| 122 | +| | SimdXml | SweetXml | Saxy | |
| 123 | +|---|---------|----------|------| |
| 124 | +| **Parser** | SIMD Rust NIF | xmerl (Erlang) | Pure Elixir SAX | |
| 125 | +| **XPath** | Full 1.0 | Full 1.0 (via xmerl) | None | |
| 126 | +| **Memory** | ~16 bytes/tag | ~350 bytes/node | Streaming | |
| 127 | +| **Atom safety** | Strings only | Creates atoms | Strings only | |
| 128 | +| **XXE safe** | No DTD processing | Vulnerable by default | No DTD processing | |
| 129 | +| **API** | Combinators + XPath | `~x` sigil | SAX handlers | |
| 130 | +| **Batch** | Bloom-filtered | No | No | |
| 131 | + |
| 132 | +## Documentation |
| 133 | + |
| 134 | +Full API docs and interactive Livebook guides: |
| 135 | + |
| 136 | +- [Getting Started](pages/getting-started.livemd) |
| 137 | +- [Query Combinators](pages/query-combinators.livemd) |
| 138 | +- [Performance Guide](pages/performance.livemd) |
| 139 | + |
| 140 | +## License |
| 141 | + |
| 142 | +MIT |
0 commit comments