diff --git a/README.md b/README.md index 2bbbec366..dce9889fb 100644 --- a/README.md +++ b/README.md @@ -75,7 +75,7 @@ Download the latest `.jar` file from: Corese supports pluggable storage backends. Create your own storage implementation (PostgreSQL, Redis, etc.) -or use built-in backends (`graph`, `memory`). +or use the default native backend (`memory`). **- [Storage Backend Plugin Guide](docs/STORAGE_PLUGIN_GUIDE.md)** diff --git a/build.gradle.kts b/build.gradle.kts index 17eabf823..8db4fa9fb 100644 --- a/build.gradle.kts +++ b/build.gradle.kts @@ -344,3 +344,16 @@ tasks.named("generateGrammarSource") { tasks.named("javaccSparqlCorese") { dependsOn("createGeneratedDirs") } + +// Standalone storage measurements use the existing test runtime, without timing JUnit. +tasks.register("storageBenchmarkClasspath") { + group = "verification" + description = "Writes the standalone storage benchmark runtime classpath." + dependsOn(tasks.testClasses) + doLast { + layout.buildDirectory.file("benchmarks/classpath.txt").get().asFile.apply { + parentFile.mkdirs() + writeText(sourceSets.test.get().runtimeClasspath.asPath) + } + } +} diff --git a/docs/STORAGE_PLUGIN_GUIDE.md b/docs/STORAGE_PLUGIN_GUIDE.md index 15a2d4aad..0ab77e03d 100644 --- a/docs/STORAGE_PLUGIN_GUIDE.md +++ b/docs/STORAGE_PLUGIN_GUIDE.md @@ -69,12 +69,9 @@ multiple providers support a configuration, Corese selects the highest priority; equal priorities are ordered by plugin name. Names must be unique and non-blank. -The built-in providers are: +The built-in provider is: -- `memory`: the default in-memory backend for tests and small datasets; -- `graph`: a compatibility adapter around the legacy Corese `Graph` backend, - requiring `graph` and `valueFactory` configuration properties. +- `memory`: the default native in-memory backend with SPO/POS/OSP indexes, + graph partitions, atomic `addAll` and session-scoped serializable snapshots. -See `GraphStoragePlugin` and `MemoryStoragePlugin` for complete provider -implementations, and `docs/source/next_storage.md` for the public storage -contract. +See `MemoryStoragePlugin` for the complete provider implementation, and `docs/source/next_storage.md` for the public storage contract. diff --git a/docs/memory-storage-final-validation.md b/docs/memory-storage-final-validation.md new file mode 100644 index 000000000..fb41df884 --- /dev/null +++ b/docs/memory-storage-final-validation.md @@ -0,0 +1,69 @@ +# Finition et validation du stockage mémoire — 14 octobre 2026 + +Les points signalés dans les deux dernières reviews sont corrigés. Aucun commit +ni push n'a été effectué. Les modifications préexistantes du nettoyage sont +conservées. + +## Timeout et transactions + +`NextSparqlPipelineExecutor` exécute toujours l'évaluation sur le thread qui possède +la transaction. Le watchdog utilise désormais un seul `AtomicInteger` : le premier +CAS départage fin d'évaluation et expiration, sans intervalle entre la revendication +d'un timeout et sa publication. Un résultat ne peut pas être accepté après que le +watchdog a gagné cette transition. + +L'échéance utilise un temps monotone mesuré à l'entrée, incluant le délai de +démarrage du watchdog. La fin d'évaluation vérifie aussi le budget, même si le +watchdog n'a pas encore été planifié. Après expiration, l'appelant attend la fin du +watchdog et de sa fermeture des ressources avant de consommer l'interruption du +timeout ; celui-ci ne peut donc plus interrompre une requête suivante. + +Deux tests couvrent l'expiration dans une transaction, sa réutilisation et son +commit, ainsi que la conservation d'une interruption externe déjà présente. +Le test existant couvre le succès de SELECT et ASK avec timeout dans une transaction. + +Les suppressions Sonar sont localisées et commentées : `S2142` pour consommer le +signal du watchdog après son arrêt, et `S9391` pour conserver la liste mutable du +dataset sans ajouter de pipeline stream à sa préparation. + +## Sondes et documentation + +Le runner actif `corese-storage-benchmarks/benchmarks/profile-memory.py` compile +les deux sondes depuis leur nouveau dépôt, avec le classpath du checkout Corese +sélectionné. Il ne cherche plus ces sources dans `corese-core/src/test`. +Il enregistre les commandes et hashes des sources des deux dépôts et utilise Java 21, +G1 et un heap de 2 Go pour le protocole de référence. Le backend actif est `memory` ; +les anciennes campagnes et leurs runtimes restent des archives. + +La documentation corrige les liens et packages déplacés, le moniteur partagé de +`Database`, le registre `State.graphs`, le seuil de huit entrées de `CompactTermMap` +et les limites de l'isolation des valeurs temporelles. Les mesures et résultats +de qualité historiques sont distingués de la validation du checkout actuel. +Le README W3C indique que les sélections autres que `memory` sont rejetées. + +## Résultats + +| Vérification | Résultat | +| --- | --- | +| Corese, `test storageBenchmarkClasspath --offline`, Java 21 | **3 155 tests**, zéro échec, erreur ou test ignoré | +| W3C, backend `memory` | **2 915 cas**, aucun changement de statut ni d'outcome | +| JUnit du harness W3C | 3 053 tests, les cinq échecs préexistants et treize tests ignorés attendus, zéro erreur | +| `validateEarlReport` | Parsing de 37 551 statements, quinze contrôles SPARQL et cohérence JSON/EARL validés | +| `enforceW3cRegressions` | Zéro régression contre la baseline officielle | +| Sonar way local, exécuteur et tests modifiés | 597 règles, deux fichiers, zéro anomalie et zéro échec d'analyse | +| Compilation stricte de l'exécuteur | `-Xlint:all -Werror`, zéro avertissement | +| Mémoire accessible inclusive, 600k, unitaire | **90 827 088 octets** | +| Mémoire accessible inclusive, 600k, batch | **90 827 088 octets** | +| Contrôle du mode heap libéré | Exécution validée sur 120 statements et trois snapshots ; ce contrôle n'est pas une mesure de performance | + +La valeur mémoire est exactement celle de la review précédente, après renommage +du backend en `memory`. Elle est inférieure de huit octets à l'ancienne référence +`indexed-memory`. Aucun fichier de production du stockage natif n'est modifié par +cette passe ; ses chemins d'ingestion et de `find()` conservent leurs sources. +Il n'a pas été lancé de nouvelle campagne chronométrée d'allocations ou de latence. + +Les résultats bruts, commandes, rapports Sonar et audits sont conservés dans +`build/cleanup-final/`. La dernière modification de JavaDoc et d'annotation de +l'exécuteur a été compilée strictement et comparée aux instructions testées : +elles sont identiques. L'analyse Sonar locale ne préjuge pas d'un profil upstream +personnalisé. diff --git a/docs/source/next_storage.md b/docs/source/next_storage.md index 283de1c42..016609d55 100644 --- a/docs/source/next_storage.md +++ b/docs/source/next_storage.md @@ -62,7 +62,56 @@ results manage that stream through their own `close()` contract. ## Transactions Call `transactions().supportsTransactions()` before beginning a transaction. -The current memory and legacy Graph adapters report no transaction support. -A future native backend may advertise its supported isolation levels and return -an `AutoCloseable` transaction handle. Closing an active transaction rolls it -back. +The `memory` backend advertises `SERIALIZABLE` and returns an `AutoCloseable` +transaction handle. Closing an active transaction rolls it back. + +## Native memory backend + +The `memory` backend is selected by default by `Storages.create()` or explicitly: + +```java +StorageManager storage = Storages.create( + StorageConfig.builder().type("memory").build()); +``` + +This implementation uses SPO, POS and OSP hash indexes in each graph partition. +Bound RDF terms select index prefixes; values are compared by RDF identity, including +literal lexical form, datatype and language. Prefix iteration is reproducible for +an identical insertion history. It supplies neither sorted value ranges nor a +SPARQL result order without `ORDER BY`. Named graphs survive `clear` and the +removal of their last statement until `dropGraph`. + +`find` captures partition snapshot references under a shared monitor and returns +a lazy stream over that snapshot without allocating an intermediate collection. +`count` and `contains` traverse the selected index without materializing the matches. +Searches across every graph visit all partitions. + +`addAll` consumes its iterable once, deduplicates quads in private staging, builds +all permutations and publishes the touched partitions together. An ordinary source +or finalization exception leaves the batch unpublished. Earlier writes in an +explicit transaction survive a failed batch. Other sessions see the batch only +after publication, or after transaction commit. A session cannot perform another +write while it consumes a batch source. Bulk construction needs temporary memory; +a smaller retained index does not necessarily imply a smaller load peak. + +Sessions share committed data and own independent transactions, even on the same +thread. Only `SERIALIZABLE` is supported. Beginning a transaction copies the graph +directory and shares its partitions (time and memory proportional to graph count). +The first write to a shared partition copies that graph's index containers; +other graphs remain shared. Autocommit writers also copy partitions still used by +snapshots. Statements and cached immutable keys remain shared. Commit verifies a +global version and publishes the private state atomically. +Any intervening committed write causes a conservative conflict, including writes +to disjoint graphs. Roll back the still-active transaction before retrying. +Transaction reads and writes must run on the creating thread. Autocommit protects +each operation; use an explicit transaction for a consistent multi-operation view. + +Closing a session rolls back its transaction. Closing the owner shuts down the +shared backend. The storage architecture and benchmarking synthesis are documented in +`corese-docs` under `Architecture/corese-storage-manager/memory-storage.md`. + +The isolation guarantees cover storage operations on RDF term identities. +Corese's built-in temporal literals expose their mutable calendar reference; +this backend preserves that API behavior. Do not mutate those calendars while +terms are in use: doing so can change temporal evaluation without a storage write. +Index snapshots share terms and do not provide deep isolation of calendar objects. diff --git a/src/main/java/fr/inria/corese/core/next/data/api/term/SimpleIRI.java b/src/main/java/fr/inria/corese/core/next/data/api/term/SimpleIRI.java index 99e475b20..504e1a5eb 100644 --- a/src/main/java/fr/inria/corese/core/next/data/api/term/SimpleIRI.java +++ b/src/main/java/fr/inria/corese/core/next/data/api/term/SimpleIRI.java @@ -4,6 +4,8 @@ import fr.inria.corese.core.next.data.api.exception.IncorrectFormatException; import java.io.Serial; +import java.io.IOException; +import java.io.ObjectInputStream; /** * Immutable default implementation of an RDF IRI. @@ -13,6 +15,10 @@ public final class SimpleIRI extends AbstractIRI { @Serial private static final long serialVersionUID = -2444556019914696994L; + // SimpleIRI is immutable. Preserve AbstractIRI's exact hash, but parse it once. + // Transient so older serialized instances rebuild the cache during deserialization. + private transient int cachedHash; + /** * Creates an IRI from its complete string representation. * @@ -21,6 +27,8 @@ public final class SimpleIRI extends AbstractIRI { */ public SimpleIRI(String fullIRI) { super(fullIRI); + // This constructor has already inferred the canonical split in AbstractIRI. + cachedHash = 31 * (31 * 7 + getNamespace().hashCode()) + getLocalName().hashCode(); } /** @@ -32,6 +40,28 @@ public SimpleIRI(String fullIRI) { */ public SimpleIRI(String namespace, String localName) { super(namespace, localName); + cachedHash = super.hashCode(); + } + + @Override + public boolean equals(Object o) { + if (this == o) { + return true; + } + if (!(o instanceof IRI that)) { + return false; + } + return stringValue().equals(that.stringValue()); } + @Override + public int hashCode() { + return cachedHash; + } + + @Serial + private void readObject(ObjectInputStream stream) throws IOException, ClassNotFoundException { + stream.defaultReadObject(); + cachedHash = super.hashCode(); + } } diff --git a/src/main/java/fr/inria/corese/core/next/data/impl/io/serializer/rdfc10/RDFC10Canonicalizer.java b/src/main/java/fr/inria/corese/core/next/data/impl/io/serializer/rdfc10/RDFC10Canonicalizer.java index 0640dccd1..a7215d020 100644 --- a/src/main/java/fr/inria/corese/core/next/data/impl/io/serializer/rdfc10/RDFC10Canonicalizer.java +++ b/src/main/java/fr/inria/corese/core/next/data/impl/io/serializer/rdfc10/RDFC10Canonicalizer.java @@ -239,12 +239,10 @@ private String hashFirstDegreeQuads(String blankNode, Map nquads = new ArrayList<>(); - for (Statement quad : quads) { - nquads.add(serializeQuad(quad, blankNode)); - } - - Collections.sort(nquads); + List nquads = quads.stream() + .map(quad -> serializeQuad(quad, blankNode)) + .sorted() + .toList(); return hash(String.join("\n", nquads) + "\n"); } diff --git a/src/main/java/fr/inria/corese/core/next/data/impl/io/serializer/rdfxml/RDFXMLSerializer.java b/src/main/java/fr/inria/corese/core/next/data/impl/io/serializer/rdfxml/RDFXMLSerializer.java index c4307ce00..d085f4688 100644 --- a/src/main/java/fr/inria/corese/core/next/data/impl/io/serializer/rdfxml/RDFXMLSerializer.java +++ b/src/main/java/fr/inria/corese/core/next/data/impl/io/serializer/rdfxml/RDFXMLSerializer.java @@ -338,7 +338,7 @@ private void writePropertyElement(Writer writer, IRI predicate, Value object, St } writer.write(currentIndent); - writer.write(String.format("<%s", elementName)); + writer.write("<" + elementName); if (object.isIRI()) { writer.write(String.format(" %s=\"%s\"/>", SerializationConstants.RDF_RESOURCE_ATTRIBUTE, escapeXmlAttribute(object.stringValue()))); @@ -371,7 +371,7 @@ private void writePropertyElement(Writer writer, IRI predicate, Value object, St writer.write(escapeXmlContent(literal.stringValue())); - writer.write(String.format("", elementName)); + writer.write(""); if(this.config instanceof LineEndingOptions lineEndingOptions) { writer.write(lineEndingOptions.getLineEnding()); } diff --git a/src/main/java/fr/inria/corese/core/next/data/impl/io/serializer/support/AbstractGraphSerializer.java b/src/main/java/fr/inria/corese/core/next/data/impl/io/serializer/support/AbstractGraphSerializer.java index d392e3a20..899c4d21c 100644 --- a/src/main/java/fr/inria/corese/core/next/data/impl/io/serializer/support/AbstractGraphSerializer.java +++ b/src/main/java/fr/inria/corese/core/next/data/impl/io/serializer/support/AbstractGraphSerializer.java @@ -119,9 +119,7 @@ protected void writeHeader(Writer writer) throws IOException { if (option instanceof BaseIRIOptions baseIRIOptions && option instanceof LineEndingOptions lineEndingOptions && baseIRIOptions.getBaseIRI() != null) { - writer.write(String.format("@base <%s> .%s", - baseIRIOptions.getBaseIRI(), - lineEndingOptions.getLineEnding())); + writer.write("@base <" + baseIRIOptions.getBaseIRI() + "> ." + lineEndingOptions.getLineEnding()); } Set actuallyUsedNamespaces = Set.of(); @@ -359,7 +357,7 @@ protected void writeIRI(Writer writer, IRI iri) throws IOException { if (prefixed != null) { writer.write(prefixed); } else { - writer.write(String.format("<%s>", escapeIRIString(iri.stringValue()))); + writer.write("<" + escapeIRIString(iri.stringValue()) + ">"); } } diff --git a/src/main/java/fr/inria/corese/core/next/query/impl/engine/eval/Eval.java b/src/main/java/fr/inria/corese/core/next/query/impl/engine/eval/Eval.java index 3f2e41709..ac3742fae 100644 --- a/src/main/java/fr/inria/corese/core/next/query/impl/engine/eval/Eval.java +++ b/src/main/java/fr/inria/corese/core/next/query/impl/engine/eval/Eval.java @@ -209,19 +209,24 @@ Mappings queryBasic(Node graphNode, Query q, Mapping m) throws SparqlException { initMemory(q); share(m); getProducer().start(q); - getVisitor().init(q); - share(getVisitor()); - getVisitor().before(q); - Mappings map = eval(graphNode, q, m, null); - getVisitor().orderby(map); - getVisitor().after(map); - - getProducer().finish(q); - if (hasEvent) { - send(Event.END, q, map); + try { + getVisitor().init(q); + share(getVisitor()); + getVisitor().before(q); + Mappings map = eval(graphNode, q, m, null); + getVisitor().orderby(map); + getVisitor().after(map); + if (hasEvent) { + send(Event.END, q, map); + } + return map; + } finally { + try { + getProducer().finish(q); + } finally { + clean(); + } } - clean(); - return map; } // share global variables and ProcessVisitor @@ -714,6 +719,11 @@ void profile(Query q) { // partial init (for global query and subqueries) public void start(Query q) { limit = q.getLimitOffset(); + // ASK only needs its first solution after OFFSET. Keep the requested + // Query modifiers intact, and evaluate fully when sorting/grouping needs it. + if (q.isAsk() && !q.isOrderBy() && !q.hasGroupBy() && !q.isAggregate()) { + limit = (int) Math.min(limit, (long) q.getOffset() + 1); + } starter(q); } @@ -938,7 +948,9 @@ int eval(Producer p, Node graphNode, Stack stack, Mappings map, int n) throws Sp break; case BGP: - backtrack = bgp(p, graphNode, exp, stack, n); + backtrack = p.evaluatesBGPWithEdges() + ? and(p, graphNode, exp, stack, map, n) + : bgp(p, graphNode, exp, stack, n); break; case SERVICE: @@ -1636,54 +1648,58 @@ private int edge(Producer p, Node graphNode, Exp exp, Mappings data, Stack stack entities = p.getEdges(graphNode, graphNodeList, qEdge, env); } - for (Edge entity : entities) { - - if (stopped) { - return STOP; - } + try { + for (Edge entity : entities) { - if (entity != null) { - nbEdge++; - if (hasListener && !listener.listen(exp, qEdge, entity)) { - continue; + if (stopped) { + return STOP; } - graph = entity.getGraph(); - boolean bmatch = match(qEdge, entity, graphNode, graph, env); + if (entity != null) { + nbEdge++; + if (hasListener && !listener.listen(exp, qEdge, entity)) { + continue; + } - if (matchNBNode) { - bmatch &= (qEdge.nbNode() == entity.nbNode()); - } + graph = entity.getGraph(); + boolean bmatch = match(qEdge, entity, graphNode, graph, env); - if (bmatch) { - if (hasCandidate) { - DatatypeValue dt = getVisitor().candidate(this, getGraphNode(graphNode), qEdge, entity); - if (dt != null) { - bmatch = dt.isTrue(); - } + if (matchNBNode) { + bmatch &= (qEdge.nbNode() == entity.nbNode()); } - bmatch &= push(p, qEdge, entity, n); - } - - if (isEvent) { - send(evENUM, exp, entity, bmatch); - } + if (bmatch) { + if (hasCandidate) { + DatatypeValue dt = getVisitor().candidate(this, getGraphNode(graphNode), qEdge, entity); + if (dt != null) { + bmatch = dt.isTrue(); + } + } - if (bmatch) { - isSuccess = true; - backtrack = eval(p, graphNode, stack, n + 1); + bmatch &= push(p, qEdge, entity, n); + } - env.pop(qEdge); - if (hasGraphNode) { - env.pop(graphNode); + if (isEvent) { + send(evENUM, exp, entity, bmatch); } - if (backtrack < n) { - return backtrack; + if (bmatch) { + isSuccess = true; + backtrack = eval(p, graphNode, stack, n + 1); + + env.pop(qEdge); + if (hasGraphNode) { + env.pop(graphNode); + } + + if (backtrack < n) { + return backtrack; + } } } } + } finally { + p.closeEdges(entities); } if (!isSuccess && optim) { diff --git a/src/main/java/fr/inria/corese/core/next/query/impl/engine/path/Visit.java b/src/main/java/fr/inria/corese/core/next/query/impl/engine/path/Visit.java index 254f562a9..fabd47d26 100644 --- a/src/main/java/fr/inria/corese/core/next/query/impl/engine/path/Visit.java +++ b/src/main/java/fr/inria/corese/core/next/query/impl/engine/path/Visit.java @@ -51,16 +51,16 @@ static class NodeTable extends TreeMap { // Traversal direction is metadata; equality must retain the Map contract. @SuppressWarnings("java:S2160") class ExpVisitedNode extends HashMap { - boolean isReverse; + boolean reverse; ExpVisitedNode(boolean b) { - isReverse = b; + reverse = b; } void add(Regex exp, Node n) { VisitedNode t = get(exp); if (t == null) { - t = new VisitedNode(isReverse, nextVisitId++); + t = new VisitedNode(reverse, nextVisitId++); put(exp, t); } t.add(n); diff --git a/src/main/java/fr/inria/corese/core/next/query/impl/engine/solution/MappingSet.java b/src/main/java/fr/inria/corese/core/next/query/impl/engine/solution/MappingSet.java index 0796cc5ca..dde7d4947 100644 --- a/src/main/java/fr/inria/corese/core/next/query/impl/engine/solution/MappingSet.java +++ b/src/main/java/fr/inria/corese/core/next/query/impl/engine/solution/MappingSet.java @@ -145,13 +145,9 @@ public boolean minusCompatible(Mapping targetMap) { } List intersectionOfUnion(MappingSet model) { - ArrayList commonVars = new ArrayList<>(); - for (String v : getUnion().keySet()) { - if (model.getUnion().containsKey(v)) { - commonVars.add(v); - } - } - return commonVars; + return getUnion().keySet().stream() + .filter(model.getUnion()::containsKey) + .toList(); } boolean inIntersection(List vars) { diff --git a/src/main/java/fr/inria/corese/core/next/query/impl/engine/spi/Producer.java b/src/main/java/fr/inria/corese/core/next/query/impl/engine/spi/Producer.java index d4eed1402..6d759cf38 100644 --- a/src/main/java/fr/inria/corese/core/next/query/impl/engine/spi/Producer.java +++ b/src/main/java/fr/inria/corese/core/next/query/impl/engine/spi/Producer.java @@ -76,6 +76,16 @@ default void finish(Query q) { */ Iterable getEdges(Node gNode, List from, Edge qEdge, Environment env); + /** Whether KGRAM should evaluate BGP edges incrementally using its regular stack. */ + default boolean evaluatesBGPWithEdges() { + return false; + } + + /** Releases an edge traversal, including when KGRAM leaves it before exhaustion. */ + default void closeEdges(Iterable edges) { + // Most producers return in-memory collections without owned resources. + } + Mappings getMappings(Node gNode, List from, Exp exp, Environment env) throws SparqlException; /* diff --git a/src/main/java/fr/inria/corese/core/next/query/impl/engine/storage/StorageManagerProducer.java b/src/main/java/fr/inria/corese/core/next/query/impl/engine/storage/StorageManagerProducer.java index 7b523b4a0..e06014c22 100644 --- a/src/main/java/fr/inria/corese/core/next/query/impl/engine/storage/StorageManagerProducer.java +++ b/src/main/java/fr/inria/corese/core/next/query/impl/engine/storage/StorageManagerProducer.java @@ -17,12 +17,20 @@ import fr.inria.corese.core.next.query.impl.engine.solution.Mappings; import fr.inria.corese.core.next.query.impl.engine.spi.Environment; import fr.inria.corese.core.next.storage.api.StorageManager; +import fr.inria.corese.core.next.storage.api.operations.StatementSpliterator; +import fr.inria.corese.core.next.storage.api.model.StatementPattern; +import fr.inria.corese.core.next.query.api.exception.QueryEvaluationException; import java.util.ArrayList; import java.util.HashSet; +import java.util.IdentityHashMap; +import java.util.Collections; +import java.util.Iterator; +import java.util.NoSuchElementException; import java.util.List; import java.util.Objects; import java.util.Set; +import java.util.Spliterators; import java.util.stream.Stream; /** @@ -36,6 +44,8 @@ public final class StorageManagerProducer extends ProducerDefault { private final StorageManager storage; + private final Set cursors = Collections.newSetFromMap(new IdentityHashMap<>()); + private boolean closed; /** * Creates a KGRAM producer backed by the given storage manager. @@ -59,19 +69,181 @@ public Iterable getEdges(Node graphNode, List from, Edge queryEdge, return List.of(); } - try (Stream statements = - storage.queries().find(queryPattern.statementPattern())) { - // A default graph formed from several contexts is an RDF graph: the - // same triple must match once, even when stored in multiple contexts. - // Deduplicate triples here, before projection, to preserve solution bags. - Set seen = new HashSet<>(); - return statements - .filter(statement -> graphNode != null || seen.add(new TripleKey( - statement.getSubject(), statement.getPredicate(), statement.getObject()))) - .map(StorageManagerEdge::new) - .map(Edge.class::cast) - .toList(); + StatementPattern pattern = queryPattern.statementPattern(); + // Statements within one explicitly selected RDF graph already form a set. + boolean deduplicate = graphNode == null && pattern.getContexts().length != 1; + return new EdgeIterable(pattern, deduplicate); + } + + /** Repeatable sequential iterable; each iteration owns one storage snapshot. */ + private final class EdgeIterable implements Iterable, AutoCloseable { + private final StatementPattern pattern; + private final boolean deduplicate; + private EdgeCursor cursor; + + private EdgeIterable(StatementPattern pattern, boolean deduplicate) { + this.pattern = pattern; + this.deduplicate = deduplicate; + } + + @Override + public Iterator iterator() { + close(); + synchronized (cursors) { + checkInterrupted(); + if (closed) { + throw new QueryEvaluationException("Query producer is closed"); + } + cursor = new EdgeCursor(storage.queries().find(pattern), deduplicate); + cursors.add(cursor); + return cursor; + } + } + + @Override + public void close() { + if (cursor != null) { + cursor.close(); + cursor = null; + } + } + } + + private final class EdgeCursor implements Iterator, AutoCloseable { + private Stream stream; + private Iterator statements; + private final Set seen; + private Statement next; + + // Release the owned snapshot before rethrowing a traversal setup failure unchanged. + @SuppressWarnings("java:S1181") + private EdgeCursor(Stream stream, boolean deduplicate) { + this.stream = stream; + try { + var source = stream.spliterator(); + // Read-version metadata, never a separate live metadata lookup: + // a writer may change the graph registry while this cursor is open. + boolean distinct = !deduplicate || source instanceof StatementSpliterator traversal + && traversal.hasDistinctTriples(); + this.seen = deduplicate && !distinct ? new HashSet<>() : null; + statements = Spliterators.iterator(source); + } catch (RuntimeException | Error failure) { + stream.close(); + throw failure; + } + } + + // Error is rethrown unchanged after releasing the snapshot, as for RuntimeException. + @Override + @SuppressWarnings("java:S1181") + public synchronized boolean hasNext() { + if (stream == null) { + return false; + } + try { + checkInterrupted(); + if (next != null) { + return true; + } + while (statements.hasNext()) { + checkInterrupted(); + Statement candidate = statements.next(); + // The merged default graph must match each RDF triple once. + if (seen == null || seen.add(new TripleKey(candidate.getSubject(), + candidate.getPredicate(), candidate.getObject()))) { + next = candidate; + return true; + } + } + close(); + return false; + } catch (RuntimeException | Error failure) { + close(); + throw failure; + } + } + + // Edge adaptation failure must also release the snapshot before propagation. + @Override + @SuppressWarnings("java:S1181") + public synchronized Edge next() { + if (!hasNext()) { + throw new NoSuchElementException(); + } + Statement statement = next; + next = null; + try { + return new StorageManagerEdge(statement); + } catch (RuntimeException | Error failure) { + close(); + throw failure; + } + } + + @Override + public synchronized void close() { + Stream current = stream; + stream = null; + statements = null; + next = null; + if (seen != null) { + seen.clear(); + } + if (current != null) { + try { + current.close(); + } finally { + synchronized (cursors) { + cursors.remove(this); + } + } + } + } + } + + private static void checkInterrupted() { + if (Thread.currentThread().isInterrupted()) { + throw new QueryEvaluationException("Query evaluation was interrupted"); + } + } + + @Override + public boolean evaluatesBGPWithEdges() { + return true; + } + + @Override + public void closeEdges(Iterable edges) { + if (edges instanceof StorageManagerProducer.EdgeIterable iterable) { + iterable.close(); + } + } + + /** Closes remaining cursors on completion, failure or cancellation. */ + private void closeCursors(boolean terminal) { + List pending; + synchronized (cursors) { + closed |= terminal; + pending = new ArrayList<>(cursors); + cursors.clear(); + } + // Never acquire a cursor monitor while holding the registry monitor: + // timeout cancellation may run concurrently with cursor traversal. + RuntimeException failure = null; + for (EdgeCursor cursor : pending) { + try { + cursor.close(); + } catch (RuntimeException exception) { + if (failure == null) failure = exception; + else failure.addSuppressed(exception); + } } + if (failure != null) throw failure; + } + + @Override + public void close() { + closeCursors(true); } private record TripleKey(Resource subject, IRI predicate, Value object) { } @@ -110,12 +282,12 @@ public Iterable getEdges( @Override public void start(Query query) { - // No per-query storage state is required. + // Cursors are registered when their iterators are opened. } @Override public void finish(Query query) { - // No per-query storage state is required. + closeCursors(false); } @Override @@ -205,13 +377,18 @@ private List join( List results = new ArrayList<>(); for (StorageBindingSet binding : inputBindings) { Environment joinedEnvironment = new StorageBindingEnvironment(environment, binding); - for (Edge candidate : getEdges(graphNode, from, queryEdge, joinedEnvironment)) { - StorageBindingSet next = binding.copy(); - if (next.bind(queryEdge.getNode(0), candidate.getNode(0)) - && next.bind(queryEdge.getNode(1), candidate.getNode(1)) - && next.bind(queryEdge.getEdgeVariable(), candidate.getEdgeNode())) { - results.add(next); + Iterable edges = getEdges(graphNode, from, queryEdge, joinedEnvironment); + try { + for (Edge candidate : edges) { + StorageBindingSet next = binding.copy(); + if (next.bind(queryEdge.getNode(0), candidate.getNode(0)) + && next.bind(queryEdge.getNode(1), candidate.getNode(1)) + && next.bind(queryEdge.getEdgeVariable(), candidate.getEdgeNode())) { + results.add(next); + } } + } finally { + closeEdges(edges); } } return results; diff --git a/src/main/java/fr/inria/corese/core/next/query/impl/query/CoreseBooleanQuery.java b/src/main/java/fr/inria/corese/core/next/query/impl/query/CoreseBooleanQuery.java index fb7865950..d3890c4c7 100644 --- a/src/main/java/fr/inria/corese/core/next/query/impl/query/CoreseBooleanQuery.java +++ b/src/main/java/fr/inria/corese/core/next/query/impl/query/CoreseBooleanQuery.java @@ -3,6 +3,7 @@ import fr.inria.corese.core.next.query.api.BooleanQuery; import fr.inria.corese.core.next.query.api.exception.QueryEvaluationException; import fr.inria.corese.core.next.query.impl.sparql.execution.NextSparqlPipelineExecutor; +import fr.inria.corese.core.next.query.impl.sparql.ast.AskQueryAst; import java.util.Objects; @@ -14,18 +15,32 @@ public final class CoreseBooleanQuery extends AbstractCoreseQuery implements BooleanQuery { private final NextSparqlPipelineExecutor executor; + private final AskQueryAst parsedQuery; public CoreseBooleanQuery( String queryString, NextSparqlPipelineExecutor executor, Runnable executionGuard) { + this(queryString, null, executor, executionGuard); + } + + /** Reuses the preparation AST while constructing fresh runtime state for each evaluation. */ + public CoreseBooleanQuery( + String queryString, + AskQueryAst parsedQuery, + NextSparqlPipelineExecutor executor, + Runnable executionGuard) { super(queryString, executionGuard); this.executor = Objects.requireNonNull(executor, "executor"); + this.parsedQuery = parsedQuery; } @Override public Boolean evaluate() throws QueryEvaluationException { checkExecutable(); - return executor.evaluateBoolean(getQueryString(), getBindings(), getDataset(), timeoutMillis()); + if (parsedQuery == null) { + return executor.evaluateBoolean(getQueryString(), getBindings(), getDataset(), timeoutMillis()); + } + return executor.evaluateBoolean(parsedQuery, getBindings(), getDataset(), timeoutMillis()); } } diff --git a/src/main/java/fr/inria/corese/core/next/query/impl/query/CoreseGraphQuery.java b/src/main/java/fr/inria/corese/core/next/query/impl/query/CoreseGraphQuery.java index a65803f5f..9e5f874b6 100644 --- a/src/main/java/fr/inria/corese/core/next/query/impl/query/CoreseGraphQuery.java +++ b/src/main/java/fr/inria/corese/core/next/query/impl/query/CoreseGraphQuery.java @@ -4,6 +4,7 @@ import fr.inria.corese.core.next.query.api.exception.QueryEvaluationException; import fr.inria.corese.core.next.query.api.result.GraphQueryResult; import fr.inria.corese.core.next.query.impl.sparql.execution.NextSparqlPipelineExecutor; +import fr.inria.corese.core.next.query.impl.sparql.ast.QueryAst; import java.util.Objects; @@ -18,18 +19,32 @@ public final class CoreseGraphQuery extends AbstractCoreseQuery implements GraphQuery { private final NextSparqlPipelineExecutor executor; + private final QueryAst parsedQuery; public CoreseGraphQuery( String queryString, NextSparqlPipelineExecutor executor, Runnable executionGuard) { + this(queryString, null, executor, executionGuard); + } + + /** Reuses the preparation AST while constructing fresh runtime state for each evaluation. */ + public CoreseGraphQuery( + String queryString, + QueryAst parsedQuery, + NextSparqlPipelineExecutor executor, + Runnable executionGuard) { super(queryString, executionGuard); this.executor = Objects.requireNonNull(executor, "executor"); + this.parsedQuery = parsedQuery; } @Override public GraphQueryResult evaluate() throws QueryEvaluationException { checkExecutable(); - return executor.evaluateGraph(getQueryString(), getBindings(), getDataset(), timeoutMillis()); + if (parsedQuery == null) { + return executor.evaluateGraph(getQueryString(), getBindings(), getDataset(), timeoutMillis()); + } + return executor.evaluateGraph(parsedQuery, getBindings(), getDataset(), timeoutMillis()); } } diff --git a/src/main/java/fr/inria/corese/core/next/query/impl/query/CoreseTupleQuery.java b/src/main/java/fr/inria/corese/core/next/query/impl/query/CoreseTupleQuery.java index 26c4a0fe5..2f80cc023 100644 --- a/src/main/java/fr/inria/corese/core/next/query/impl/query/CoreseTupleQuery.java +++ b/src/main/java/fr/inria/corese/core/next/query/impl/query/CoreseTupleQuery.java @@ -4,6 +4,7 @@ import fr.inria.corese.core.next.query.api.exception.QueryEvaluationException; import fr.inria.corese.core.next.query.api.result.TupleQueryResult; import fr.inria.corese.core.next.query.impl.sparql.execution.NextSparqlPipelineExecutor; +import fr.inria.corese.core.next.query.impl.sparql.ast.SelectQueryAst; import java.util.Objects; @@ -17,18 +18,32 @@ public final class CoreseTupleQuery extends AbstractCoreseQuery implements TupleQuery { private final NextSparqlPipelineExecutor executor; + private final SelectQueryAst parsedQuery; public CoreseTupleQuery( String queryString, NextSparqlPipelineExecutor executor, Runnable executionGuard) { + this(queryString, null, executor, executionGuard); + } + + /** Reuses the preparation AST while constructing fresh runtime state for each evaluation. */ + public CoreseTupleQuery( + String queryString, + SelectQueryAst parsedQuery, + NextSparqlPipelineExecutor executor, + Runnable executionGuard) { super(queryString, executionGuard); this.executor = Objects.requireNonNull(executor, "executor"); + this.parsedQuery = parsedQuery; } @Override public TupleQueryResult evaluate() throws QueryEvaluationException { checkExecutable(); - return executor.evaluateTuple(getQueryString(), getBindings(), getDataset(), timeoutMillis()); + if (parsedQuery == null) { + return executor.evaluateTuple(getQueryString(), getBindings(), getDataset(), timeoutMillis()); + } + return executor.evaluateTuple(parsedQuery, getBindings(), getDataset(), timeoutMillis()); } } diff --git a/src/main/java/fr/inria/corese/core/next/query/impl/repository/CoreseRepositoryConnection.java b/src/main/java/fr/inria/corese/core/next/query/impl/repository/CoreseRepositoryConnection.java index 464ffd1fb..eff080a46 100644 --- a/src/main/java/fr/inria/corese/core/next/query/impl/repository/CoreseRepositoryConnection.java +++ b/src/main/java/fr/inria/corese/core/next/query/impl/repository/CoreseRepositoryConnection.java @@ -229,11 +229,11 @@ public TupleQuery prepareTupleQuery(String queryString, String baseIRI) { checkOpen(); String source = requireSource(queryString, PARAM_QUERY_STRING); QueryAst ast = parser.parse(source, baseIRI); - if (!(ast instanceof SelectQueryAst)) { + if (!(ast instanceof SelectQueryAst select)) { throw new QuerySyntaxException( "Expected a SELECT query, got: " + ast.getClass().getSimpleName()); } - CoreseTupleQuery q = new CoreseTupleQuery(source, executorFor(baseIRI), this::checkOpen); + CoreseTupleQuery q = new CoreseTupleQuery(source, select, executorFor(baseIRI), this::checkOpen); applyConnectionDataset(q); return q; } @@ -253,7 +253,7 @@ public GraphQuery prepareGraphQuery(String queryString, String baseIRI) { throw new QuerySyntaxException( "Expected a CONSTRUCT or DESCRIBE query, got: " + ast.getClass().getSimpleName()); } - CoreseGraphQuery q = new CoreseGraphQuery(source, executorFor(baseIRI), this::checkOpen); + CoreseGraphQuery q = new CoreseGraphQuery(source, ast, executorFor(baseIRI), this::checkOpen); applyConnectionDataset(q); return q; } @@ -269,11 +269,11 @@ public BooleanQuery prepareBooleanQuery(String queryString, String baseIRI) { checkOpen(); String source = requireSource(queryString, PARAM_QUERY_STRING); QueryAst ast = parser.parse(source, baseIRI); - if (!(ast instanceof AskQueryAst)) { + if (!(ast instanceof AskQueryAst ask)) { throw new QuerySyntaxException( "Expected an ASK query, got: " + ast.getClass().getSimpleName()); } - CoreseBooleanQuery q = new CoreseBooleanQuery(source, executorFor(baseIRI), this::checkOpen); + CoreseBooleanQuery q = new CoreseBooleanQuery(source, ask, executorFor(baseIRI), this::checkOpen); applyConnectionDataset(q); return q; } diff --git a/src/main/java/fr/inria/corese/core/next/query/impl/sparql/execution/NextSparqlPipelineExecutor.java b/src/main/java/fr/inria/corese/core/next/query/impl/sparql/execution/NextSparqlPipelineExecutor.java index 3a68b959c..9c8c5b303 100644 --- a/src/main/java/fr/inria/corese/core/next/query/impl/sparql/execution/NextSparqlPipelineExecutor.java +++ b/src/main/java/fr/inria/corese/core/next/query/impl/sparql/execution/NextSparqlPipelineExecutor.java @@ -44,10 +44,8 @@ import java.util.Map; import java.util.Objects; import java.util.Set; -import java.util.concurrent.ExecutionException; -import java.util.concurrent.FutureTask; import java.util.concurrent.TimeUnit; -import java.util.concurrent.TimeoutException; +import java.util.concurrent.atomic.AtomicInteger; /** * Internal orchestrator for the Corese-next SPARQL query path. @@ -64,6 +62,10 @@ */ public final class NextSparqlPipelineExecutor { + private static final int QUERY_RUNNING = 0; + private static final int QUERY_COMPLETED = 1; + private static final int QUERY_TIMED_OUT = 2; + private final StorageManager storage; private final SparqlParser parser; private final CoreseAstQueryBuilder queryBuilder; @@ -134,13 +136,17 @@ public TupleQueryResult evaluateTuple(String sparql, BindingSet bindings, Datase throw new IllegalArgumentException("Tuple evaluation requires a SELECT query, got: " + ast.getClass().getSimpleName()); } - return new CoreseTupleQueryResult( - evaluate(queryBuilder.toNextQuery(select), bindings, dataset, timeoutMillis)); + return evaluateTuple(select, bindings, dataset, timeoutMillis); } /** Evaluates an already parsed SELECT, also used by SPARQL Update WHERE clauses. */ public TupleQueryResult evaluateTuple(SelectQueryAst query) { - return new CoreseTupleQueryResult(evaluate(queryBuilder.toNextQuery(query), null, null, 0L)); + return evaluateTuple(query, null, null, 0L); + } + + /** Evaluates a parsed SELECT with current options and a fresh runtime query. */ + public TupleQueryResult evaluateTuple(SelectQueryAst query, BindingSet bindings, Dataset dataset, long timeoutMillis) { + return new CoreseTupleQueryResult(evaluate(queryBuilder.toNextQuery(query), bindings, dataset, timeoutMillis)); } /** @@ -174,7 +180,12 @@ public boolean evaluateBoolean(String sparql, BindingSet bindings, Dataset datas throw new IllegalArgumentException("Boolean evaluation requires an ASK query, got: " + ast.getClass().getSimpleName()); } - return evaluate(queryBuilder.toNextQuery(ask), bindings, dataset, timeoutMillis).size() > 0; + return evaluateBoolean(ask, bindings, dataset, timeoutMillis); + } + + /** Evaluates a parsed ASK with current options and a fresh runtime query. */ + public boolean evaluateBoolean(AskQueryAst query, BindingSet bindings, Dataset dataset, long timeoutMillis) { + return evaluate(queryBuilder.toNextQuery(query), bindings, dataset, timeoutMillis).size() > 0; } /** @@ -208,7 +219,12 @@ public GraphQueryResult evaluateGraph(String sparql) { * @throws QueryTimeoutException when the evaluation exceeds {@code timeoutMillis} */ public GraphQueryResult evaluateGraph(String sparql, BindingSet bindings, Dataset dataset, long timeoutMillis) { - QueryAst ast = parser.parse(sparql); + return evaluateGraph(parser.parse(sparql), bindings, dataset, timeoutMillis); + } + + /** Evaluates a parsed CONSTRUCT or DESCRIBE with a fresh runtime query and template. */ + public GraphQueryResult evaluateGraph(QueryAst ast, BindingSet bindings, Dataset dataset, long timeoutMillis) { + Objects.requireNonNull(ast, "ast"); Query kgramQuery = switch (ast) { case ConstructQueryAst construct -> queryBuilder.toNextQuery(construct); case DescribeQueryAst describe -> queryBuilder.toNextQuery(describe); @@ -226,8 +242,9 @@ public GraphQueryResult evaluateGraph(String sparql, BindingSet bindings, Datase // ------------------------------------------------------------------------- private Mappings evaluate(Query kgramQuery, BindingSet bindings, Dataset dataset, long timeoutMillis) { + StorageManagerProducer producer = new StorageManagerProducer(storage); Eval eval = Eval.create( - new StorageManagerProducer(storage), + producer, new SparqlKgramEvaluator(), new RdfTermMatcher()); @@ -238,7 +255,7 @@ private Mappings evaluate(Query kgramQuery, BindingSet bindings, Dataset dataset Mapping initialMapping = buildInitialMapping(bindings); if (timeoutMillis > 0) { - return evaluateWithTimeout(eval, kgramQuery, initialMapping, timeoutMillis); + return evaluateWithTimeout(eval, producer, kgramQuery, initialMapping, timeoutMillis); } return evaluateCore(eval, kgramQuery, initialMapping); } @@ -253,38 +270,85 @@ private Mappings evaluateCore(Eval eval, Query kgramQuery, Mapping initialMappin } /** - * Runs {@code eval.query()} with a cooperative timeout. + * Runs {@code eval.query()} with a cooperative timeout on the caller thread. * - *

The evaluation runs in a request-scoped virtual thread. At the deadline, - * {@link Eval#finish()} cooperatively stops KGRAM and the task is interrupted. - * No process-wide scheduler or mutable global timeout state is retained.

+ *

Evaluation runs on the current thread so active storage transactions maintain + * single-thread ownership. A request-scoped virtual thread watchdog cooperatively terminates + * evaluation and interrupts the caller if {@code timeoutMillis} elapses. Completion + * and expiration compete for one atomic terminal state; the timer cannot publish + * an expiration separately from claiming it. Elapsed time is measured from entry, + * including watchdog scheduling delay.

*/ - private Mappings evaluateWithTimeout(Eval eval, Query kgramQuery, Mapping initialMapping, long timeoutMillis) { - FutureTask evaluation = new FutureTask<>( - () -> evaluateCore(eval, kgramQuery, initialMapping)); - Thread thread = Thread.ofVirtual().name("sparql-query").start(evaluation); + private Mappings evaluateWithTimeout(Eval eval, StorageManagerProducer producer, Query kgramQuery, Mapping initialMapping, long timeoutMillis) { + Thread caller = Thread.currentThread(); + boolean interruptedOnEntry = caller.isInterrupted(); + long started = System.nanoTime(); + long budget = TimeUnit.MILLISECONDS.toNanos(timeoutMillis); + AtomicInteger outcome = new AtomicInteger(QUERY_RUNNING); + Thread watchdog = Thread.ofVirtual().name("sparql-timeout-watchdog").start(() -> { + try { + long remaining = budget - (System.nanoTime() - started); + if (remaining > 0) { + TimeUnit.NANOSECONDS.sleep(remaining); + } + if (outcome.compareAndSet(QUERY_RUNNING, QUERY_TIMED_OUT)) { + caller.interrupt(); + eval.finish(); + producer.close(); + } + } catch (InterruptedException ignored) { + Thread.currentThread().interrupt(); + } + }); + try { - return evaluation.get(timeoutMillis, TimeUnit.MILLISECONDS); - } catch (TimeoutException exception) { - eval.finish(); - evaluation.cancel(true); - throw new QueryTimeoutException(timeoutMillis); - } catch (InterruptedException exception) { - eval.finish(); - evaluation.cancel(true); - Thread.currentThread().interrupt(); - throw new QueryEvaluationException("Query evaluation was interrupted", exception); - } catch (ExecutionException exception) { - Throwable cause = exception.getCause(); - if (cause instanceof QueryEvaluationException queryFailure) { - throw queryFailure; + Mappings mappings = evaluateCore(eval, kgramQuery, initialMapping); + if (!outcome.compareAndSet(QUERY_RUNNING, QUERY_COMPLETED) + || System.nanoTime() - started >= budget) { + throw new QueryTimeoutException(timeoutMillis); } - throw new QueryEvaluationException("Query evaluation failed", cause); + return mappings; + } catch (RuntimeException exception) { + if (!outcome.compareAndSet(QUERY_RUNNING, QUERY_COMPLETED) + && outcome.get() == QUERY_TIMED_OUT + && !(exception instanceof QueryTimeoutException)) { + throw new QueryTimeoutException(timeoutMillis); + } + throw exception; } finally { - if (thread.isAlive() && evaluation.isCancelled()) { - thread.interrupt(); + outcome.compareAndSet(QUERY_RUNNING, QUERY_COMPLETED); + finishWatchdog(watchdog, outcome.get() == QUERY_TIMED_OUT, interruptedOnEntry); + } + } + + /** + * Cancels a pending timer, or waits for timeout interruption and resource cleanup to finish. + * A timeout's interruption is consumed only after the watchdog terminates, so it cannot + * interrupt a later query. An interruption already present on entry remains set. + * + * @param watchdog request-scoped timeout thread + * @param timedOut whether expiration won the atomic outcome transition + * @param interruptedOnEntry whether the caller entered evaluation interrupted + */ + @SuppressWarnings("java:S2142") // Consume watchdog interruption after joining; restore the caller's entry status. + private static void finishWatchdog(Thread watchdog, boolean timedOut, boolean interruptedOnEntry) { + if (!timedOut) { + watchdog.interrupt(); + return; + } + boolean joined = false; + while (!joined) { + try { + watchdog.join(); + joined = true; + } catch (InterruptedException interruption) { + // The watchdog interrupts this caller; finish waiting before consuming that signal. } } + Thread.interrupted(); + if (interruptedOnEntry) { + Thread.currentThread().interrupt(); + } } // ------------------------------------------------------------------------- @@ -301,17 +365,24 @@ private Mappings evaluateWithTimeout(Eval eval, Query kgramQuery, Mapping initia * RDF term are silently skipped, matching standard SPARQL CONSTRUCT semantics.

*/ private List buildConstructStatements(Query kgramQuery, Mappings mappings) { - List templateEdges = new ArrayList<>(); Exp constructTemplate = kgramQuery.getConstruct(); if (constructTemplate == null) { return List.of(); } + List templateEdges = new ArrayList<>(); constructTemplate.getEdgeList(templateEdges); ValueFactory factory = Values.factory(); List statements = new ArrayList<>(); + Set usedBlankLabels = collectUsedBlankLabels(mappings); + + for (Mapping mapping : mappings) { + instantiateTemplateMapping(templateEdges, mapping, usedBlankLabels, factory, statements); + } + return statements; + } - // Reserve every solution’s labels before instantiation, including later solutions. + private Set collectUsedBlankLabels(Mappings mappings) { Set usedBlankLabels = new HashSet<>(); for (Mapping mapping : mappings) { for (Node node : mapping.getNodes()) { @@ -320,28 +391,33 @@ private List buildConstructStatements(Query kgramQuery, Mappings mapp } } } - for (Mapping mapping : mappings) { - // Fresh blank node map per solution: blank-variable label → constant blank node - Map blankNodeMap = new HashMap<>(); - for (Edge templateEdge : templateEdges) { - Node subjectNode = resolveTemplateNode(templateEdge.getNode(0), mapping, blankNodeMap, usedBlankLabels); - Node predicateNode = resolveTemplateNode(templateEdge.getProperty(), mapping, blankNodeMap, usedBlankLabels); - Node objectNode = resolveTemplateNode(templateEdge.getNode(1), mapping, blankNodeMap, usedBlankLabels); - - if (subjectNode == null || predicateNode == null || objectNode == null) { - continue; - } + return usedBlankLabels; + } - Value subject = kgramNodeToApiValue(subjectNode); - Value predicate = kgramNodeToApiValue(predicateNode); - Value object = kgramNodeToApiValue(objectNode); + private void instantiateTemplateMapping( + List templateEdges, + Mapping mapping, + Set usedBlankLabels, + ValueFactory factory, + List statements) { + Map blankNodeMap = new HashMap<>(); + for (Edge templateEdge : templateEdges) { + Node subjectNode = resolveTemplateNode(templateEdge.getNode(0), mapping, blankNodeMap, usedBlankLabels); + Node predicateNode = resolveTemplateNode(templateEdge.getProperty(), mapping, blankNodeMap, usedBlankLabels); + Node objectNode = resolveTemplateNode(templateEdge.getNode(1), mapping, blankNodeMap, usedBlankLabels); + + if (subjectNode == null || predicateNode == null || objectNode == null) { + continue; + } - if (subject instanceof Resource s && predicate instanceof IRI p && object != null) { - statements.add(factory.createStatement(s, p, object)); - } + Value subject = kgramNodeToApiValue(subjectNode); + Value predicate = kgramNodeToApiValue(predicateNode); + Value object = kgramNodeToApiValue(objectNode); + + if (subject instanceof Resource s && predicate instanceof IRI p && object != null) { + statements.add(factory.createStatement(s, p, object)); } } - return statements; } /** @@ -408,6 +484,8 @@ private void applyDataset(Query kgramQuery, Dataset dataset) { kgramQuery.setDatasetSpecified(true); } + /** Translates dataset IRIs in encounter order into KGRAM's mutable dataset list. */ + @SuppressWarnings("java:S9391") // Keep the existing list allocation without a stream pipeline for this iterable. private List irisToKgramNodes(Iterable iris) { List nodes = new ArrayList<>(); for (IRI iri : iris) { diff --git a/src/main/java/fr/inria/corese/core/next/query/impl/sparql/parser/SparqlAstBuilder.java b/src/main/java/fr/inria/corese/core/next/query/impl/sparql/parser/SparqlAstBuilder.java index d555a0ad6..afee85e9c 100644 --- a/src/main/java/fr/inria/corese/core/next/query/impl/sparql/parser/SparqlAstBuilder.java +++ b/src/main/java/fr/inria/corese/core/next/query/impl/sparql/parser/SparqlAstBuilder.java @@ -56,6 +56,11 @@ public abstract class SparqlAstBuilder { */ protected final Deque> bgpStack = new ArrayDeque<>(); + // TriplesBlock is right-recursive: exit callbacks arrive in reverse source order. + // Reserve each block's position on entry instead of appending it on exit. + private record BgpPosition(List group, int index) { } + private final Deque bgpPositions = new ArrayDeque<>(); + /** * Counter for anonymous blank nodes created while expanding {@code [ ... ]} and {@code ( ... )} subjects. */ @@ -332,6 +337,9 @@ public void addFromNamedGraph(IriAst graph) { */ public void enterBgp() { bgpStack.push(new ArrayList<>()); + List group = currentGroup(); + bgpPositions.push(new BgpPosition(group, group.size())); + group.add(new BgpAst(List.of())); } /** @@ -340,9 +348,9 @@ public void enterBgp() { */ public void exitBgp() { List triples = bgpStack.pop(); - if (!triples.isEmpty()) { - currentGroup().add(new BgpAst(List.copyOf(triples))); - } + BgpPosition position = bgpPositions.pop(); + if (triples.isEmpty()) position.group().remove(position.index()); + else position.group().set(position.index(), new BgpAst(triples)); } /** @@ -757,16 +765,14 @@ public void addTriplesFromPropertyListPath( int verbSimpleIdx = 0; for (SparqlParser.ObjectListPathContext objectList : objectLists) { - PathAst predicate = predicateFromPropertyListPath( - verbPaths, verbSimples, verbPathIdx, verbSimpleIdx); - if (verbPathIdx < verbPaths.size() + boolean useVerbPath = verbPathIdx < verbPaths.size() && (verbSimpleIdx >= verbSimples.size() || verbPaths.get(verbPathIdx).getStart().getTokenIndex() - < verbSimples.get(verbSimpleIdx).getStart().getTokenIndex())) { - verbPathIdx++; - } else { - verbSimpleIdx++; - } + < verbSimples.get(verbSimpleIdx).getStart().getTokenIndex()); + + PathAst predicate = useVerbPath + ? pathFromVerbPath(verbPaths.get(verbPathIdx++)) + : new PredicatePathAst(terms.termFromVerbSimple(verbSimples.get(verbSimpleIdx++))); for (TermAst object : termListFromObjectListPath(objectList)) { addTriple(subject, predicate, object); @@ -774,21 +780,6 @@ public void addTriplesFromPropertyListPath( } } - private PathAst predicateFromPropertyListPath( - List verbPaths, - List verbSimples, - int verbPathIdx, - int verbSimpleIdx) { - boolean useVerbPath = verbPathIdx < verbPaths.size() - && (verbSimpleIdx >= verbSimples.size() - || verbPaths.get(verbPathIdx).getStart().getTokenIndex() - < verbSimples.get(verbSimpleIdx).getStart().getTokenIndex()); - if (useVerbPath) { - return pathFromVerbPath(verbPaths.get(verbPathIdx)); - } - return new PredicatePathAst(terms.termFromVerbSimple(verbSimples.get(verbSimpleIdx))); - } - /** * Builds triple patterns for a {@code triplesNodePath} subject and returns its head term. */ diff --git a/src/main/java/fr/inria/corese/core/next/query/impl/sparql/parser/SparqlQueryAnalyzer.java b/src/main/java/fr/inria/corese/core/next/query/impl/sparql/parser/SparqlQueryAnalyzer.java index f3ddf3aaa..18e73ee94 100644 --- a/src/main/java/fr/inria/corese/core/next/query/impl/sparql/parser/SparqlQueryAnalyzer.java +++ b/src/main/java/fr/inria/corese/core/next/query/impl/sparql/parser/SparqlQueryAnalyzer.java @@ -26,6 +26,7 @@ import java.util.LinkedHashSet; import java.util.List; import java.util.Set; +import java.util.stream.Collectors; /** * Performs SPARQL syntax, AST, and semantic analysis and returns the @@ -190,13 +191,10 @@ private QueryAst buildAst( private Set blankNodeLabels(CommonTokenStream tokens) { tokens.fill(); - Set labels = new LinkedHashSet<>(); - for (Token token : tokens.getTokens()) { - if (token.getType() == SparqlLexer.BLANK_NODE_LABEL) { - labels.add(token.getText()); - } - } - return labels; + return tokens.getTokens().stream() + .filter(token -> token.getType() == SparqlLexer.BLANK_NODE_LABEL) + .map(Token::getText) + .collect(Collectors.toCollection(LinkedHashSet::new)); } private QueryDiagnostic toSyntaxDiagnostic(QuerySyntaxException exception) { diff --git a/src/main/java/fr/inria/corese/core/next/query/impl/sparql/parser/SparqlTermBuilder.java b/src/main/java/fr/inria/corese/core/next/query/impl/sparql/parser/SparqlTermBuilder.java index 7f848f387..7cc77d6ed 100644 --- a/src/main/java/fr/inria/corese/core/next/query/impl/sparql/parser/SparqlTermBuilder.java +++ b/src/main/java/fr/inria/corese/core/next/query/impl/sparql/parser/SparqlTermBuilder.java @@ -34,7 +34,8 @@ public final class SparqlTermBuilder { /** * Constructs a {@code SparqlTermBuilder} with a blank-node resolver callback. * - * @param options parser options (base IRI, etc.) + * @param options parser options validated for non-null configuration + * (reserved for future term resolution extensions) * @param blankNodeResolver callback that handles {@code BlankNodeContext} and * updates the mutable blank-node label set held by * {@link SparqlAstBuilder} @@ -105,34 +106,43 @@ public LiteralAst literal(String lexical, String lang, String datatype) { */ private static void validateNoSurrogateUchars(String lexical) { int i = 0; - while (i < lexical.length()) { - char c = lexical.charAt(i); - if (c == '\\' && i + 1 < lexical.length()) { - char next = lexical.charAt(i + 1); - if (next == '\\') { - i += 2; - continue; - } - int hexLen = 0; - if (next == 'u') hexLen = 4; - else if (next == 'U') hexLen = 8; - if (hexLen > 0 && i + 2 + hexLen <= lexical.length()) { - String hex = lexical.substring(i + 2, i + 2 + hexLen); - try { - int codePoint = Integer.parseInt(hex, 16); - if (codePoint >= 0xD800 && codePoint <= 0xDFFF) { - throw new QuerySyntaxException( - "Surrogate code point U+" + hex.toUpperCase() - + " is not a valid Unicode scalar value in a SPARQL escape sequence"); - } - } catch (NumberFormatException ignored) { - // Not a valid hex sequence; ANTLR grammar ensures this won't happen for UCHAR - } - i += 2 + hexLen; - continue; - } + int len = lexical.length(); + while (i < len) { + i += checkEscapeAt(lexical, i, len); + } + } + + private static int checkEscapeAt(String lexical, int i, int len) { + if (lexical.charAt(i) != '\\' || i + 1 >= len) { + return 1; + } + char next = lexical.charAt(i + 1); + if (next == '\\') { + return 2; + } + int hexLen = 0; + if (next == 'u') { + hexLen = 4; + } else if (next == 'U') { + hexLen = 8; + } + if (hexLen > 0 && i + 2 + hexLen <= len) { + checkSurrogateHex(lexical.substring(i + 2, i + 2 + hexLen)); + return 2 + hexLen; + } + return 1; + } + + private static void checkSurrogateHex(String hex) { + try { + int codePoint = Integer.parseInt(hex, 16); + if (codePoint >= 0xD800 && codePoint <= 0xDFFF) { + throw new QuerySyntaxException( + "Surrogate code point U+" + hex.toUpperCase() + + " is not a valid Unicode scalar value in a SPARQL escape sequence"); } - i++; + } catch (NumberFormatException ignored) { + // Not a valid hex sequence; ANTLR grammar ensures this won't happen for UCHAR } } diff --git a/src/main/java/fr/inria/corese/core/next/storage/api/operations/StatementSpliterator.java b/src/main/java/fr/inria/corese/core/next/storage/api/operations/StatementSpliterator.java new file mode 100644 index 000000000..1f3518164 --- /dev/null +++ b/src/main/java/fr/inria/corese/core/next/storage/api/operations/StatementSpliterator.java @@ -0,0 +1,34 @@ +package fr.inria.corese.core.next.storage.api.operations; + +import fr.inria.corese.core.next.data.api.model.Statement; + +import java.util.Spliterator; + +/** + * Optional snapshot-bound guarantees for a statement traversal. + * + *

+ * A storage stream may expose this interface through {@link java.util.stream.Stream#spliterator()}. Wrappers and + * backends without this interface require conservative handling by callers. + *

+ */ +public interface StatementSpliterator extends Spliterator { + + /** + * Whether this traversal contains each RDF subject/predicate/object triple at most once, ignoring its context. The + * guarantee must stay valid throughout this traversal, including across concurrent mutations of the underlying + * storage. + * + *

+ * This is stronger than {@link Spliterator#DISTINCT}, which may only distinguish statements by their contexts. A + * single RDF graph normally provides this guarantee; a union of graph partitions may not. + *

+ * + *

The guarantee must describe the same captured read version as this traversal. + * Inspecting it must not consume statements or change snapshot ownership. + * An implementation may return false when uniqueness cannot be established.

+ * + * @return true only when triple uniqueness is guaranteed for the captured read + */ + boolean hasDistinctTriples(); +} diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/graph/CoreseGraphStatementStore.java b/src/main/java/fr/inria/corese/core/next/storage/impl/graph/CoreseGraphStatementStore.java deleted file mode 100644 index f94bf34dd..000000000 --- a/src/main/java/fr/inria/corese/core/next/storage/impl/graph/CoreseGraphStatementStore.java +++ /dev/null @@ -1,404 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.graph; - -import fr.inria.corese.core.Graph; -import fr.inria.corese.core.kgram.api.core.Edge; -import fr.inria.corese.core.kgram.api.core.Node; -import fr.inria.corese.core.next.data.api.factory.ValueFactory; -import fr.inria.corese.core.next.data.api.model.Statement; -import fr.inria.corese.core.next.data.api.term.BNode; -import fr.inria.corese.core.next.data.api.term.IRI; -import fr.inria.corese.core.next.data.api.term.Literal; -import fr.inria.corese.core.next.data.api.term.Resource; -import fr.inria.corese.core.next.data.api.term.Value; -import fr.inria.corese.core.sparql.api.IDatatype; - -import java.util.Arrays; -import java.util.HashSet; -import java.util.Optional; -import java.util.Set; - -/** - * Adapter between legacy {@link Graph} and new Storage API. - */ -record CoreseGraphStatementStore(Graph graph, ValueFactory valueFactory) { - - /** - * Compact constructor with validation. - * - * @throws IllegalArgumentException if graph or valueFactory is null - */ - public CoreseGraphStatementStore { - if (graph == null) throw new IllegalArgumentException("Graph cannot be null"); - if (valueFactory == null) throw new IllegalArgumentException("ValueFactory cannot be null"); - } - - /** - * Adds a statement to the underlying Graph. - * - * @param stmt the statement to add - * @return true if the statement was added, false if it already existed - */ - public boolean add(Statement stmt) { - // Convert statement components to nodes - Node subject = resourceToNode(stmt.getSubject()); - Node predicate = iriToNode(stmt.getPredicate()); - Node object = valueToNode(stmt.getObject()); - - Resource ctxResource = stmt.getContext(); - Node context = (ctxResource != null) - ? resourceToNode(ctxResource) - : graph.getDefaultGraphNode(); - - // Always use 4-argument form with explicit context - Edge edge = graph.addEdge(context, subject, predicate, object); - - return edge != null; - } - - /** - * Removes a statement from the underlying Graph. - * - * @param stmt the statement to remove - * @return true (Graph.delete() does not return a boolean) - */ - public boolean remove(Statement stmt) { - Edge edge = statementToEdge(stmt); - graph.delete(edge); - return true; - } - - /** - * Finds all statements matching the given pattern. - * - * @param s subject filter, or null for any - * @param p predicate filter, or null for any - * @param o object filter, or null for any - * @param contexts context filters; null/empty = wildcard, {null} = default graph only - * @return set of matching statements - */ - public Set find(Resource s, IRI p, Value o, Resource[] contexts) { - Node subject = s != null ? resourceToNode(s) : null; - Node predicate = p != null ? iriToNode(p) : null; - Node object = o != null ? valueToNode(o) : null; - - Iterable edges; - if (contexts == null || contexts.length == 0) { - edges = graph.getEdgesRDF4J(subject, predicate, object); - } else { - // Normalize contexts: convert null to default graph node - Node[] ctxNodes = normalizeContexts(contexts); - edges = graph.getEdgesRDF4J(subject, predicate, object, ctxNodes); - } - - Set results = new HashSet<>(); - for (Edge edge : edges) { - if (edge != null) { - try { - results.add(edgeToStatement(edge)); - } catch (IllegalArgumentException e) { - // Skip edges that cannot be converted - } - } - } - return results; - } - - /** - * Returns the total number of statements in the Graph. - * - * @return statement count - */ - public int size() { - return graph.size(); - } - - /** - * Removes all statements from the Graph. - */ - public void clear() { - graph.clear(); - } - - /** - * Removes all statements from a specific named graph. - * - * @param context the named graph to clear - */ - public void clearContext(Resource context) { - Node ctx = resourceToNode(context); - graph.clear(ctx.getLabel()); - } - - /** - * Returns all unique subject resources in the Graph. - * - * @return set of subjects - */ - public Set getSubjects() { - Set subjects = new HashSet<>(); - for (Edge edge : graph.getEdges()) { - if (edge != null && edge.getSubjectNode() != null) { - try { - subjects.add(nodeToResource(edge.getSubjectNode())); - } catch (IllegalArgumentException e) { - // Skip invalid nodes - } - } - } - return subjects; - } - - /** - * Returns all unique predicate IRIs in the Graph. - * - * @return set of predicates - */ - public Set getPredicates() { - Set predicates = new HashSet<>(); - for (Edge edge : graph.getEdges()) { - if (edge != null && edge.getEdgeNode() != null) { - try { - predicates.add(nodeToIRI(edge.getEdgeNode())); - } catch (IllegalArgumentException e) { - // Skip invalid nodes - } - } - } - return predicates; - } - - /** - * Returns all unique object values in the Graph. - * - * @return set of objects - */ - public Set getObjects() { - Set objects = new HashSet<>(); - for (Edge edge : graph.getEdges()) { - if (edge != null && edge.getObjectNode() != null) { - try { - objects.add(nodeToValue(edge.getObjectNode())); - } catch (IllegalArgumentException e) { - // Skip invalid nodes - } - } - } - return objects; - } - - /** - * Returns all unique named graph identifiers in the Graph. - * - * @return set of context resources - */ - public Set getContexts() { - Set contexts = new HashSet<>(); - for (Node ctx : graph.getGraphNodes()) { - if (ctx != null) { - try { - contexts.add(nodeToResource(ctx)); - } catch (IllegalArgumentException e) { - // Skip invalid nodes - } - } - } - return contexts; - } - - /** - * Normalizes context array for Graph backend. - * - * @param contexts the context resources (may contain null) - * @return normalized array of Graph nodes (never contains null) - */ - private Node[] normalizeContexts(Resource[] contexts) { - return Arrays.stream(contexts) - .map(ctx -> ctx != null ? resourceToNode(ctx) : graph.getDefaultGraphNode()) - .toArray(Node[]::new); - } - - /** - * Converts a Graph {@link Edge} to a Storage {@link Statement}. - * - * @param edge the Graph edge to convert - * @return the corresponding Statement - * @throws IllegalArgumentException if edge or any of its required nodes is null - */ - private Statement edgeToStatement(Edge edge) { - if (edge == null) { - throw new IllegalArgumentException("Edge cannot be null"); - } - - Node subjectNode = edge.getSubjectNode(); - Node predicateNode = edge.getEdgeNode(); - Node objectNode = edge.getObjectNode(); - - if (subjectNode == null) { - throw new IllegalArgumentException("Edge subject node is null"); - } - if (predicateNode == null) { - throw new IllegalArgumentException("Edge predicate node is null"); - } - if (objectNode == null) { - throw new IllegalArgumentException("Edge object node is null"); - } - - Resource subject = nodeToResource(subjectNode); - IRI predicate = nodeToIRI(predicateNode); - Value object = nodeToValue(objectNode); - - Node graphNode = edge.getGraph(); - Resource context = (graphNode == null || graph.isDefaultGraphNode(graphNode)) - ? null - : nodeToResource(graphNode); - - if (context == null) { - return valueFactory.createStatement(subject, predicate, object); - } else { - return valueFactory.createStatement(subject, predicate, object, context); - } - } - - /** - * Converts a Graph {@link Node} to a Storage {@link Resource} (IRI or BNode). - */ - private Resource nodeToResource(Node node) { - if (node == null) { - throw new IllegalArgumentException("Node cannot be null"); - } - - IDatatype dt = node.getDatatypeValue(); - if (dt == null) { - throw new IllegalArgumentException("Node datatype is null: " + node); - } - - if (dt.isURI()) { - return valueFactory.createIRI(dt.getLabel()); - } else if (dt.isBlank()) { - return valueFactory.createBNode(dt.getLabel()); - } else if (dt.isTriple()) { - return valueFactory.createBNode(dt.getLabel()); - } else { - throw new IllegalArgumentException("Node is not a Resource: " + node); - } - } - - /** - * Converts a Graph {@link Node} (predicate) to a Storage {@link IRI}. - */ - private IRI nodeToIRI(Node node) { - if (node == null) { - throw new IllegalArgumentException("Node cannot be null"); - } - - IDatatype dt = node.getDatatypeValue(); - if (dt == null) { - throw new IllegalArgumentException("Node datatype is null: " + node); - } - - if (!dt.isURI()) { - throw new IllegalArgumentException("Node is not an IRI: " + node); - } - - return valueFactory.createIRI(dt.getLabel()); - } - - /** - * Converts a Graph {@link Node} to a Storage {@link Value} (Resource or Literal). - */ - private Value nodeToValue(Node node) { - if (node == null) { - throw new IllegalArgumentException("Node cannot be null"); - } - - IDatatype dt = node.getDatatypeValue(); - if (dt == null) { - throw new IllegalArgumentException("Node datatype is null: " + node); - } - - if (dt.isURI() || dt.isBlank() || dt.isTriple()) { - return nodeToResource(node); - } else if (dt.isLiteral()) { - String label = dt.getLabel(); - String lang = dt.getLang(); - String datatypeIRI = dt.getDatatypeURI(); - - if (lang != null && !lang.isEmpty()) { - return valueFactory.createLiteral(label, lang); - } else if (datatypeIRI != null && !datatypeIRI.isEmpty()) { - IRI datatype = valueFactory.createIRI(datatypeIRI); - return valueFactory.createLiteral(label, datatype); - } else { - return valueFactory.createLiteral(label); - } - } else { - throw new IllegalArgumentException("Unknown node type: " + node); - } - } - - /** - * Converts a Storage {@link Statement} to a Graph {@link Edge}. - * - * @param stmt the Statement to convert - * @return the corresponding Edge - */ - private Edge statementToEdge(Statement stmt) { - Node subject = resourceToNode(stmt.getSubject()); - Node predicate = iriToNode(stmt.getPredicate()); - Node object = valueToNode(stmt.getObject()); - - Resource ctxResource = stmt.getContext(); - Node context = (ctxResource != null) - ? resourceToNode(ctxResource) - : graph.getDefaultGraphNode(); - - return graph.create(context, subject, predicate, object); - } - - /** - * Converts a Storage {@link Resource} to a Graph {@link Node}. - */ - private Node resourceToNode(Resource resource) { - if (resource.isIRI()) { - IRI iri = (IRI) resource; - return graph.addResource(iri.stringValue()); - } else if (resource.isBNode()) { - BNode bnode = (BNode) resource; - return graph.addBlank(bnode.getID()); - } else { - throw new IllegalArgumentException("Unknown Resource type: " + resource); - } - } - - /** - * Converts a Storage {@link IRI} to a Graph {@link Node} (property). - */ - private Node iriToNode(IRI iri) { - return graph.addProperty(iri.stringValue()); - } - - /** - * Converts a Storage {@link Value} to a Graph {@link Node}. - */ - private Node valueToNode(Value value) { - if (value.isResource()) { - return resourceToNode((Resource) value); - } else if (value.isLiteral()) { - Literal literal = (Literal) value; - String label = literal.getLabel(); - - Optional langOpt = literal.getLanguage(); - if (langOpt.isPresent()) { - String lang = langOpt.get(); - return graph.addLiteral(label, null, lang); - } else if (literal.getDatatype() != null) { - String datatypeIRI = literal.getDatatype().stringValue(); - return graph.addLiteral(label, datatypeIRI); - } else { - return graph.addLiteral(label); - } - } else { - throw new IllegalArgumentException("Unknown Value type: " + value); - } - } -} diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/graph/GraphLifecycleManager.java b/src/main/java/fr/inria/corese/core/next/storage/impl/graph/GraphLifecycleManager.java deleted file mode 100644 index a1a8ca715..000000000 --- a/src/main/java/fr/inria/corese/core/next/storage/impl/graph/GraphLifecycleManager.java +++ /dev/null @@ -1,94 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.graph; - -import fr.inria.corese.core.Graph; -import fr.inria.corese.core.next.storage.api.lifecycle.LifecycleState; -import fr.inria.corese.core.next.storage.api.lifecycle.StorageLifecycle; -import fr.inria.corese.core.next.storage.api.config.StorageConfig; -import fr.inria.corese.core.next.storage.api.exception.ErrorCode; -import fr.inria.corese.core.next.storage.api.exception.StorageException; - -import java.util.Optional; -import java.util.concurrent.atomic.AtomicReference; - -/** - * Lifecycle manager for {@link GraphStorageManager}. - */ -final class GraphLifecycleManager implements StorageLifecycle { - - private final Graph graph; - private volatile LifecycleState state = LifecycleState.NOT_INITIALIZED; - private final AtomicReference config = new AtomicReference<>(); - - /** - * Constructs a new GraphLifecycleManager. - * - * @param graph the Graph to manage (must not be null) - * @throws IllegalArgumentException if graph is null - */ - public GraphLifecycleManager(Graph graph) { - if (graph == null) throw new IllegalArgumentException("Graph cannot be null"); - this.graph = graph; - } - - /** - * Initializes the Graph with the given configuration. - * - * @param config the storage configuration (must not be null) - * @throws IllegalArgumentException if config is null - * @throws IllegalStateException if already initialized - * @throws StorageException if Graph initialization fails - */ - @Override - public synchronized void initialize(StorageConfig config) throws StorageException { - if (config == null) throw new IllegalArgumentException("Config cannot be null"); - if (state != LifecycleState.NOT_INITIALIZED) { - throw new IllegalStateException("Already initialized"); - } - - this.state = LifecycleState.INITIALIZING; - try { - graph.init(); - this.config.set(config); - this.state = LifecycleState.RUNNING; - } catch (Exception e) { - this.state = LifecycleState.NOT_INITIALIZED; - throw new StorageException(ErrorCode.INITIALIZATION_FAILED, "Init failed", e); - } - } - - /** - * Shuts down the Graph. - * - * @throws IllegalStateException if not currently running - * @throws StorageException if shutdown fails - */ - @Override - public synchronized void shutdown() throws StorageException { - if (state != LifecycleState.RUNNING) { - throw new IllegalStateException("Not running"); - } - this.state = LifecycleState.SHUTTING_DOWN; - this.state = LifecycleState.SHUTDOWN; - this.config.set(null); - } - - /** - * Returns the current lifecycle state. - * - * @return the current state (never null) - */ - @Override - public LifecycleState getState() { - return state; - } - - /** - * Returns the current configuration. - * - * @return the config if initialized/running, empty otherwise - */ - @Override - public Optional getConfig() { - return Optional.ofNullable(config.get()); - } -} diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/graph/GraphMetadataOperations.java b/src/main/java/fr/inria/corese/core/next/storage/impl/graph/GraphMetadataOperations.java deleted file mode 100644 index a6f14e595..000000000 --- a/src/main/java/fr/inria/corese/core/next/storage/impl/graph/GraphMetadataOperations.java +++ /dev/null @@ -1,88 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.graph; - -import fr.inria.corese.core.next.data.api.term.IRI; -import fr.inria.corese.core.next.data.api.term.Resource; -import fr.inria.corese.core.next.data.api.term.Value; -import fr.inria.corese.core.next.storage.api.operations.MetadataOperations; -import fr.inria.corese.core.next.storage.api.exception.StorageException; -import fr.inria.corese.core.next.storage.api.model.StorageStatistics; - -import java.util.Collections; -import java.util.Set; - -/** - * Metadata operations implementation for {@link GraphStorageManager}. - */ -final class GraphMetadataOperations implements MetadataOperations { - - private final CoreseGraphStatementStore adapter; - - /** - * Constructs a new GraphMetadataOperations. - * - * @param adapter the CoreseGraphStatementStore for Graph access (must not be null) - * @throws IllegalArgumentException if adapter is null - */ - public GraphMetadataOperations(CoreseGraphStatementStore adapter) { - if (adapter == null) throw new IllegalArgumentException("CoreseGraphStatementStore cannot be null"); - this.adapter = adapter; - } - - /** - * Returns all unique predicate IRIs in the Graph. - * - * @return unmodifiable set of all predicates - * @throws StorageException if the operation fails - */ - @Override - public Set getPredicates() throws StorageException { - return Collections.unmodifiableSet(adapter.getPredicates()); - } - - /** - * Returns all unique subject resources in the Graph. - * - * @return unmodifiable set of all subjects - */ - @Override - public Set getSubjects() { - return Collections.unmodifiableSet(adapter.getSubjects()); - } - - /** - * Returns all unique object values in the Graph. - * - * @return unmodifiable set of all objects - */ - @Override - public Set getObjects() { - return Collections.unmodifiableSet(adapter.getObjects()); - } - - /** - * Returns all unique context (named graph) identifiers in the Graph. - * - * @return unmodifiable set of all contexts - * @throws StorageException if the operation fails - */ - @Override - public Set getContexts() throws StorageException { - return Collections.unmodifiableSet(adapter.getContexts()); - } - - /** - * Returns aggregate statistics about the Graph. - * - * @return a {@link StorageStatistics} record with aggregate counts - */ - @Override - public StorageStatistics getStatistics() { - return new StorageStatistics( - adapter.size(), - adapter.getSubjects().size(), - adapter.getPredicates().size(), - adapter.getObjects().size(), - adapter.getContexts().size() - ); - } -} diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/graph/GraphMutationOperations.java b/src/main/java/fr/inria/corese/core/next/storage/impl/graph/GraphMutationOperations.java deleted file mode 100644 index d09306a10..000000000 --- a/src/main/java/fr/inria/corese/core/next/storage/impl/graph/GraphMutationOperations.java +++ /dev/null @@ -1,146 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.graph; - -import fr.inria.corese.core.next.data.api.model.Statement; -import fr.inria.corese.core.next.data.api.term.Resource; -import fr.inria.corese.core.next.storage.api.operations.MutationOperations; -import fr.inria.corese.core.next.storage.api.exception.ErrorCode; -import fr.inria.corese.core.next.storage.api.exception.StorageException; -import fr.inria.corese.core.next.storage.api.model.StatementPattern; - -import java.util.Objects; - -/** - * Mutation operations implementation for {@link GraphStorageManager}. - */ -final class GraphMutationOperations implements MutationOperations { - - private final CoreseGraphStatementStore adapter; - - /** - * Constructs a new GraphMutationOperations. - * - * @param adapter the CoreseGraphStatementStore for Graph access (must not be null) - * @throws IllegalArgumentException if adapter is null - */ - public GraphMutationOperations(CoreseGraphStatementStore adapter) { - if (adapter == null) throw new IllegalArgumentException("CoreseGraphStatementStore cannot be null"); - this.adapter = adapter; - } - - /** - * Creates an empty named graph in the underlying Corese graph. - * - * @param graph the named graph identifier (must not be {@code null}) - * @return {@code true} if the graph was created, {@code false} if it already existed - * @throws NullPointerException if {@code graph} is {@code null} - */ - @Override - public boolean createGraph(Resource graph) { - Objects.requireNonNull(graph, "graph"); - if (adapter.getContexts().contains(graph)) { - return false; - } - adapter.graph().addGraph(graph.stringValue(), graph.isBNode()); - return true; - } - - /** - * Drops a named graph and clears all its statements. - * - * @param graph the named graph identifier (must not be {@code null}) - * @return {@code true} if the graph was removed, {@code false} if it was absent - * @throws NullPointerException if {@code graph} is {@code null} - */ - @Override - public boolean dropGraph(Resource graph) { - Objects.requireNonNull(graph, "graph"); - if (!adapter.getContexts().contains(graph)) { - return false; - } - clear(graph); - adapter.graph().deleteGraph(graph.stringValue()); - return true; - } - - /** - * Inserts a statement into the Graph. - * - * @param statement the statement to insert (must not be null) - * @return whether the graph changed - * @throws NullPointerException if statement is null - * @throws StorageException if the insert operation fails - */ - @Override - public boolean add(Statement statement) throws StorageException { - Objects.requireNonNull(statement, "statement"); - try { - return adapter.add(statement); - } catch (Exception e) { - throw new StorageException(ErrorCode.MUTATION_FAILED, "Insert failed", e); - } - } - - /** - * Deletes a statement from the Graph. - * - * @param statement the statement to delete (must not be null) - * @return whether the graph changed - * @throws NullPointerException if statement is null - * @throws StorageException if the delete operation fails - */ - @Override - public boolean remove(Statement statement) throws StorageException { - Objects.requireNonNull(statement, "statement"); - try { - return adapter.remove(statement); - } catch (Exception e) { - throw new StorageException(ErrorCode.MUTATION_FAILED, "Delete failed", e); - } - } - - /** - * Deletes all statements matching the given pattern. - * - * @param pattern statement pattern - * @return number of deleted statements - * @throws StorageException if the operation fails - */ - @Override - public long remove(StatementPattern pattern) throws StorageException { - Objects.requireNonNull(pattern, "pattern"); - try { - var toRemove = adapter.find( - pattern.getSubject(), - pattern.getPredicate(), - pattern.getObject(), - pattern.getContexts()); - long count = toRemove.size(); - toRemove.forEach(adapter::remove); - return count; - } catch (Exception e) { - throw new StorageException(ErrorCode.MUTATION_FAILED, "Delete failed", e); - } - } - - /** - * Clears statements from specified contexts. - * - * @param contexts contexts to clear; null/empty clears every graph and an - * explicit null element clears the default graph - * @return number of deleted statements - * @throws StorageException if the clear operation fails - */ - @Override - public long clear(Resource... contexts) throws StorageException { - try { - if (contexts != null && contexts.length > 0) { - return remove(StatementPattern.of(null, null, null, contexts)); - } - int before = adapter.size(); - adapter.clear(); - return (long) before - adapter.size(); - } catch (Exception e) { - throw new StorageException(ErrorCode.MUTATION_FAILED, "Clear failed", e); - } - } -} diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/graph/GraphQueryOperations.java b/src/main/java/fr/inria/corese/core/next/storage/impl/graph/GraphQueryOperations.java deleted file mode 100644 index 6bf913605..000000000 --- a/src/main/java/fr/inria/corese/core/next/storage/impl/graph/GraphQueryOperations.java +++ /dev/null @@ -1,79 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.graph; - -import fr.inria.corese.core.next.data.api.model.Statement; -import fr.inria.corese.core.next.storage.api.operations.QueryOperations; -import fr.inria.corese.core.next.storage.api.exception.ErrorCode; -import fr.inria.corese.core.next.storage.api.exception.StorageException; -import fr.inria.corese.core.next.storage.api.model.StatementPattern; - -import java.util.Objects; -import java.util.stream.Stream; - -/** - * Query operations implementation for {@link GraphStorageManager}. - */ -final class GraphQueryOperations implements QueryOperations { - - private final CoreseGraphStatementStore adapter; - - /** - * Constructs a new GraphQueryOperations. - * - * @param adapter the CoreseGraphStatementStore for Graph access (must not be null) - * @throws NullPointerException if adapter is null - */ - public GraphQueryOperations(CoreseGraphStatementStore adapter) { - this.adapter = Objects.requireNonNull(adapter, "adapter"); - } - - /** - * Queries for statements matching the given pattern. - * - * @param pattern the statement pattern to match (must not be null) - * @return a stream of matching statements (must be closed) - * @throws NullPointerException if pattern is null - * @throws StorageException if the query fails - */ - @Override - public Stream find(StatementPattern pattern) throws StorageException { - Objects.requireNonNull(pattern, "pattern"); - try { - return adapter.find( - pattern.getSubject(), - pattern.getPredicate(), - pattern.getObject(), - pattern.getContexts() - ).stream(); - } catch (Exception e) { - throw new StorageException(ErrorCode.QUERY_FAILED, "Query failed", e); - } - } - - /** - * Counts the number of statements matching the given pattern. - * - * @param pattern the statement pattern to match (must not be null) - * @return the count of matching statements - * @throws StorageException if the query fails - */ - @Override - public long count(StatementPattern pattern) throws StorageException { - try (Stream stream = find(pattern)) { - return stream.count(); - } - } - - /** - * Checks if any statement matches the given pattern. - * - * @param pattern the statement pattern to match (must not be null) - * @return true if at least one matching statement exists - * @throws StorageException if the query fails - */ - @Override - public boolean contains(StatementPattern pattern) throws StorageException { - try (Stream stream = find(pattern)) { - return stream.findAny().isPresent(); - } - } -} diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/graph/GraphStorageManager.java b/src/main/java/fr/inria/corese/core/next/storage/impl/graph/GraphStorageManager.java deleted file mode 100644 index 524e5b53f..000000000 --- a/src/main/java/fr/inria/corese/core/next/storage/impl/graph/GraphStorageManager.java +++ /dev/null @@ -1,123 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.graph; - -import fr.inria.corese.core.Graph; -import fr.inria.corese.core.next.data.api.factory.ValueFactory; -import fr.inria.corese.core.next.storage.api.StorageManager; -import fr.inria.corese.core.next.storage.api.lifecycle.StorageLifecycle; -import fr.inria.corese.core.next.storage.api.operations.MetadataOperations; -import fr.inria.corese.core.next.storage.api.operations.MutationOperations; -import fr.inria.corese.core.next.storage.api.operations.QueryOperations; -import fr.inria.corese.core.next.storage.api.transaction.TransactionManager; - -/** - * StorageManager implementation wrapping the legacy {@link Graph} class. - * - */ -public final class GraphStorageManager implements StorageManager { - - private final CoreseGraphStatementStore adapter; - private final QueryOperations queryOps; - private final MutationOperations mutationOps; - private final MetadataOperations metadataOps; - private final TransactionManager txManager; - private final StorageLifecycle lifecycle; - - private GraphStorageManager(Builder builder) { - if (builder.graph == null) { - throw new IllegalArgumentException("Graph cannot be null"); - } - if (builder.valueFactory == null) { - throw new IllegalArgumentException("ValueFactory cannot be null"); - } - - this.adapter = new CoreseGraphStatementStore(builder.graph, builder.valueFactory); - this.queryOps = new GraphQueryOperations(adapter); - this.mutationOps = new GraphMutationOperations(adapter); - this.metadataOps = new GraphMetadataOperations(adapter); - this.txManager = new GraphTransactionManager(); - this.lifecycle = new GraphLifecycleManager(builder.graph); - } - - @Override - public QueryOperations queries() { - return queryOps; - } - - @Override - public MutationOperations mutations() { - return mutationOps; - } - - @Override - public MetadataOperations metadata() { - return metadataOps; - } - - @Override - public TransactionManager transactions() { - return txManager; - } - - @Override - public StorageLifecycle lifecycle() { - return lifecycle; - } - - /** - * Returns the underlying legacy Graph instance. - * - * @return the wrapped Graph - */ - public Graph getGraph() { - return adapter.graph(); - } - - public static Builder builder() { - return new Builder(); - } - - /** - * Builder for GraphStorageManager. - */ - public static final class Builder { - private Graph graph; - private ValueFactory valueFactory; - - /** - * Sets the legacy Graph instance to wrap. - * - * @param graph the Graph instance (must not be null) - * @return this builder - */ - public Builder graph(Graph graph) { - this.graph = graph; - return this; - } - - /** - * Sets the ValueFactory for creating Storage API objects. - * - * @param valueFactory the factory (must not be null) - * @return this builder - */ - public Builder valueFactory(ValueFactory valueFactory) { - this.valueFactory = valueFactory; - return this; - } - - public GraphStorageManager build() { - if (graph == null) { - throw new IllegalStateException("Graph must be set before building"); - } - if (valueFactory == null) { - throw new IllegalStateException("ValueFactory must be set before building"); - } - return new GraphStorageManager(this); - } - } - - @Override - public String toString() { - return "GraphStorageManager{size=" + adapter.size() + "}"; - } -} diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/graph/GraphTransactionManager.java b/src/main/java/fr/inria/corese/core/next/storage/impl/graph/GraphTransactionManager.java deleted file mode 100644 index ed196230b..000000000 --- a/src/main/java/fr/inria/corese/core/next/storage/impl/graph/GraphTransactionManager.java +++ /dev/null @@ -1,55 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.graph; - -import fr.inria.corese.core.next.storage.api.transaction.IsolationLevel; -import fr.inria.corese.core.next.storage.api.transaction.Transaction; -import fr.inria.corese.core.next.storage.api.transaction.TransactionManager; - -import java.util.Collections; -import java.util.Set; - -/** - * No-op transaction manager for {@link GraphStorageManager}. - */ -final class GraphTransactionManager implements TransactionManager { - - /** - * Always returns false as legacy Graph does not support transactions. - * - * @return false - */ - @Override - public boolean supportsTransactions() { - return false; - } - - /** - * Attempts to begin a transaction. - * - * @return never returns (always throws) - */ - @Override - public Transaction beginTransaction() { - throw new UnsupportedOperationException("Transactions not supported by legacy Graph"); - } - - /** - * Attempts to begin a transaction with a specific isolation level. - * - * @param level the requested isolation level (ignored) - * @return never returns (always throws) - */ - @Override - public Transaction beginTransaction(IsolationLevel level) { - throw new UnsupportedOperationException("Transactions not supported by legacy Graph"); - } - - /** - * Returns the set of supported isolation levels. - * - * @return empty set - */ - @Override - public Set getSupportedIsolationLevels() { - return Collections.emptySet(); - } -} diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/graph/package-info.java b/src/main/java/fr/inria/corese/core/next/storage/impl/graph/package-info.java deleted file mode 100644 index 554759f16..000000000 --- a/src/main/java/fr/inria/corese/core/next/storage/impl/graph/package-info.java +++ /dev/null @@ -1,7 +0,0 @@ -/** - * Corese Graph-backed storage implementation. - * - *

All classes here depend on the legacy Corese {@code Graph} type. That - * dependency must not leak into the storage manager API or other impl packages.

- */ -package fr.inria.corese.core.next.storage.impl.graph; diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/graph/plugin/GraphStoragePlugin.java b/src/main/java/fr/inria/corese/core/next/storage/impl/graph/plugin/GraphStoragePlugin.java deleted file mode 100644 index f12724162..000000000 --- a/src/main/java/fr/inria/corese/core/next/storage/impl/graph/plugin/GraphStoragePlugin.java +++ /dev/null @@ -1,69 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.graph.plugin; - -import fr.inria.corese.core.Graph; -import fr.inria.corese.core.next.data.api.factory.ValueFactory; -import fr.inria.corese.core.next.storage.api.StorageManager; -import fr.inria.corese.core.next.storage.api.plugin.PluginException; -import fr.inria.corese.core.next.storage.api.plugin.StoragePlugin; -import fr.inria.corese.core.next.storage.api.config.StorageConfig; -import fr.inria.corese.core.next.storage.impl.graph.GraphStorageManager; - -import java.util.Objects; - -/** - * Plugin for GraphStorageManager - wraps legacy Corese Graph backend. - */ -public class GraphStoragePlugin implements StoragePlugin { - - private static final String PLUGIN_NAME = "graph"; - - @Override - public String getName() { - return PLUGIN_NAME; - } - - @Override - public String getDescription() { - return "Compatibility adapter for the legacy Corese Graph backend"; - } - - @Override - public boolean supports(StorageConfig config) { - if (config == null) { - return false; - } - return config.getType() - .map(PLUGIN_NAME::equalsIgnoreCase) - .orElse(false); - } - - @Override - public StorageManager create(StorageConfig config) throws PluginException { - StorageConfig checkedConfig = Objects.requireNonNull(config, "config"); - try { - Graph graph = checkedConfig.getProperty(PLUGIN_NAME, Graph.class) - .orElseThrow(() -> new PluginException("Graph instance required in config properties")); - - ValueFactory factory = checkedConfig.getProperty("valueFactory", ValueFactory.class) - .orElseThrow(() -> new PluginException("ValueFactory required in config properties")); - - GraphStorageManager storage = GraphStorageManager.builder() - .graph(graph) - .valueFactory(factory) - .build(); - - storage.lifecycle().initialize(checkedConfig); - - return storage; - } catch (PluginException e) { - throw e; - } catch (Exception e) { - throw new PluginException("Failed to create GraphStorageManager", e); - } - } - - @Override - public int getPriority() { - return 100; - } -} diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/graph/plugin/package-info.java b/src/main/java/fr/inria/corese/core/next/storage/impl/graph/plugin/package-info.java deleted file mode 100644 index 256f1bc2f..000000000 --- a/src/main/java/fr/inria/corese/core/next/storage/impl/graph/plugin/package-info.java +++ /dev/null @@ -1,2 +0,0 @@ -/** Service provider for the legacy Corese graph storage backend. */ -package fr.inria.corese.core.next.storage.impl.graph.plugin; diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/CompactTermMap.java b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/CompactTermMap.java new file mode 100644 index 000000000..5f76b93c7 --- /dev/null +++ b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/CompactTermMap.java @@ -0,0 +1,563 @@ +package fr.inria.corese.core.next.storage.impl.memory; + +import java.util.Arrays; +import java.util.Objects; +import java.util.function.BiConsumer; +import java.util.function.Function; +import java.util.function.Predicate; +import java.util.function.ToLongFunction; + +/** + * Insertion-ordered term map with representations adapted to small cardinalities. + * + *

+ * An empty map owns no entry buffer. A singleton stores its key separately and its value directly in {@code entries}. + * Larger small maps use alternating key and value slots in an {@code Object[]} array. Beyond the array threshold, + * {@code entries} holds a {@link DenseTermTable}. Dense maps demote at half the promotion threshold to avoid + * oscillation. Keys and values must be non-null; representation transitions and filtering preserve the order of + * survivors. + * + *

+ * Copies preserve the current representation and duplicate its mutable buffers. The mapped copy constructor + * additionally copies nested mutable values, allowing index branches to own their leaves while sharing immutable terms + * and statements. Mutation requires exclusive partition ownership under the database monitor. Positional traversal + * requires an unchanged, retained partition for its lifetime. + * + * @param non-null entry value type + */ +class CompactTermMap { + /** + * Maximum flat-pair cardinality before dense promotion. + */ + private static final int ARRAY_LIMIT = 8; + /** + * Key used only by the singleton representation. + */ + private TermKey singletonKey; + /** + * Empty, singleton, flat-pair or dense representation of live values. + */ + private Object entries; + /** + * Number of live entries or statements owned by this container. + */ + private int size; + + /** + * Creates an empty compact map; buffers are allocated on demand. + */ + CompactTermMap() { + } + + /** + * Copies the container buffers while sharing keys and values. + * + * @param source source index or container; it must remain unchanged during copying or traversal + */ + + CompactTermMap(final CompactTermMap source) { + singletonKey = source.singletonKey; + size = source.size; + if (source.entries instanceof DenseTermTable) { + entries = new DenseTermTable<>(source.table()); + } else if (size > 1) { + entries = ((Object[]) source.entries).clone(); + } else { + entries = source.entries; + } + } + + /** + * Preserve the representation and its independent buffers; map values without lookups. + * + * @param source source index or container; it must remain unchanged during copying or traversal + * @param values mapper for live values in the copied, independently owned buffers + * @throws NullPointerException if an explicitly required argument or mapped value is null + */ + + CompactTermMap(final CompactTermMap source, final Function values) { + this(source); + if (entries instanceof DenseTermTable) { + table().mapValues(values); + } else if (size > 1) { + final Object[] pairs = (Object[]) entries; + for (int i = 1; i < size * 2; i += 2) { + pairs[i] = Objects.requireNonNull(values.apply(value(pairs[i]))); + } + } else if (size == 1) { + entries = Objects.requireNonNull(values.apply(value(entries))); + } + } + + /** + * Returns the number of live entries. + * + * @return live entry count + */ + int size() { + return size; + } + + /** + * Reports whether this container has no live entries. + * + * @return true when no live entries remain + */ + boolean isEmpty() { + return size == 0; + } + + /** + * Reads the sole value of a singleton representation. + * + * @return the sole non-null value + * @throws IllegalStateException if the required representation, ownership or lifecycle state is not satisfied + */ + V singletonValue() { + if (size != 1) { + throw new IllegalStateException("Not a singleton"); + } + return value(entries); + } + + /** + * Replaces the value of an existing key without changing its position. + * + * @param key search or insertion key; null denotes a wildcard only for pattern traversal + * @param replacement non-null replacement for an existing value + * @throws IllegalStateException if the required representation, ownership or lifecycle state is not satisfied + */ + void replace(final TermKey key, final V replacement) { + if (size == 1 && same(key, singletonKey)) { + entries = replacement; + return; + } + if (entries instanceof DenseTermTable) { + if (table().replace(key, replacement) == null) { + throw new IllegalStateException("Missing key"); + } + return; + } + if (size > 1) { + final Object[] pairs = (Object[]) entries; + for (int i = 0; i < size * 2; i += 2) { + if (same(key, (TermKey) pairs[i])) { + pairs[i + 1] = replacement; + return; + } + } + } + throw new IllegalStateException("Missing key"); + } + + /** + * Reads an immutable RDF value or a value held in the container representation. + * + * @param value non-null RDF value or entry value + * @return the stored RDF value or the non-null entry value at the specified position + */ + @SuppressWarnings("unchecked") + private V value(final Object value) { + return (V) value; + } + + /** + * Accesses the dense representation; the caller must have established that it is active. + * + * @return the active dense representation + */ + @SuppressWarnings("unchecked") + private DenseTermTable table() { + return (DenseTermTable) entries; + } + + /** + * Compares term keys using their cached hashes before structural equality. + * + * @param left first canonical term key + * @param right second canonical term key + * @return true when both cached hashes and RDF identities match + */ + private static boolean same(final TermKey left, final TermKey right) { + return left.hashCode() == right.hashCode() && left.equals(right); + } + + /** + * Looks up a value without changing encounter order. + * + * @param candidate search key + * @return the matching value, or null if absent + */ + V get(final TermKey candidate) { + if (size == 0) { + return null; + } + if (size == 1) { + return same(candidate, singletonKey) ? value(entries) : null; + } + if (entries instanceof DenseTermTable) { + return table().get(candidate); + } + final Object[] pairs = (Object[]) entries; + for (int i = 0; i < size * 2; i += 2) { + if (same(candidate, (TermKey) pairs[i])) { + return value(pairs[i + 1]); + } + } + return null; + } + + /** + * Adds a statement or keyed value if it is absent. + * + * @param key search or insertion key; null denotes a wildcard only for pattern traversal + * @param value non-null RDF value or entry value + * @return true when an absent entry was added, false when it was already present + */ + boolean add(final TermKey key, final V value) { + if (size == 0) { + singletonKey = key; + entries = value; + size = 1; + return true; + } + if (size == 1) { + if (same(key, singletonKey)) { + return false; + } + entries = new Object[] { singletonKey, entries, key, value }; + singletonKey = null; + size = 2; + return true; + } + if (entries instanceof DenseTermTable) { + if (table().putIfAbsent(key, value) != null) { + return false; + } + size++; + return true; + } + return addToArray(key, value); + } + + /** + * Adds an absent pair to the flat representation, promoting at the existing threshold. + * + * @param key search or insertion key; null denotes a wildcard only for pattern traversal + * @param value non-null RDF value or entry value + * @return true after insertion, false for an existing key + */ + private boolean addToArray(final TermKey key, final V value) { + if (get(key) != null) { + return false; + } + Object[] pairs = (Object[]) entries; + if (size == ARRAY_LIMIT) { + final DenseTermTable table = new DenseTermTable<>(size + 1); + for (int i = 0; i < size * 2; i += 2) { + table.put((TermKey) pairs[i], value(pairs[i + 1])); + } + table.put(key, value); + entries = table; + } else { + if (pairs.length == size * 2) { + entries = pairs = Arrays.copyOf(pairs, pairs.length * 2); + } + pairs[size * 2] = key; + pairs[size * 2 + 1] = value; + } + size++; + return true; + } + + /** + * Removes matching content while preserving the order of survivors. + * + * @param key search or insertion key; null denotes a wildcard only for pattern traversal + * @return true when an entry was removed, false when it was absent + */ + boolean remove(final TermKey key) { + if (size == 0) { + return false; + } + if (size == 1) { + if (!same(key, singletonKey)) { + return false; + } + singletonKey = null; + entries = null; + size = 0; + return true; + } + return entries instanceof DenseTermTable ? removeFromTable(key) : removeFromArray(key); + } + + /** + * Removes a key from the dense representation and demotes when its threshold is reached. + * + * @param key search or insertion key; null denotes a wildcard only for pattern traversal + * @return true after removal, false for an absent key + */ + private boolean removeFromTable(final TermKey key) { + if (table().remove(key) == null) { + return false; + } + size--; + if (size <= ARRAY_LIMIT / 2) { + final Object[] pairs = new Object[size * 2]; + int offset = 0; + for (var entry : table().entrySet()) { + pairs[offset++] = entry.getKey(); + pairs[offset++] = entry.getValue(); + } + entries = pairs; + } + return true; + } + + /** + * Removes a pair from the flat representation with stable shifts and existing shrink rules. + * + * @param key search or insertion key; null denotes a wildcard only for pattern traversal + * @return true after removal, false for an absent key + */ + private boolean removeFromArray(final TermKey key) { + final Object[] pairs = (Object[]) entries; + for (int i = 0; i < size * 2; i += 2) { + if (!same(key, (TermKey) pairs[i])) { + continue; + } + removePairAt(i, pairs); + return true; + } + return false; + } + + /** + * Removes one live flat pair, preserving survivor order and the existing shrink thresholds. + * + * @param index even offset of the matching live key + * @param pairs exclusively owned flat key/value buffer + */ + private void removePairAt(final int index, final Object[] pairs) { + System.arraycopy(pairs, index + 2, pairs, index, (size * 2) - index - 2); + pairs[(size * 2) - 2] = null; + pairs[(size * 2) - 1] = null; + size--; + if (size == 1) { + singletonKey = (TermKey) pairs[0]; + entries = pairs[1]; + } else if (size * 4 <= pairs.length) { + entries = Arrays.copyOf(pairs, size * 2); + } + } + + /** + * Visits the selected value or all values in encounter order, stopping when the visitor returns true. + * + * @param key search or insertion key; null denotes a wildcard only for pattern traversal + * @param visitor non-mutating callback; true requests an early stop for predicate visitors + * @return true when a selected value caused the visitor to request an early stop + */ + boolean visitMatching(final TermKey key, final Predicate visitor) { + if (key != null) { + final V match = get(key); + return match != null && visitor.test(match); + } + if (size == 0) { + return false; + } + if (size == 1) { + return visitor.test(value(entries)); + } + if (entries instanceof DenseTermTable) { + return table().visit(visitor); + } + final Object[] pairs = (Object[]) entries; + for (int i = 1; i < size * 2; i += 2) { + if (visitor.test(value(pairs[i]))) { + return true; + } + } + return false; + } + + /** + * Delete many values in insertion order without repeated array shifts. + * + * @param predicate non-mutating predicate or captured predicate key, according to the operation + * @return number of removed entries + */ + + int removeIf(final Predicate predicate) { + if (size == 0) { + return 0; + } + if (size == 1) { + if (!predicate.test(value(entries))) { + return 0; + } + singletonKey = null; + entries = null; + size = 0; + return 1; + } + final int previous = size; + Object[] pairs; + if (entries instanceof DenseTermTable) { + final int removed = table().removeIf(predicate); + if (removed == 0) { + return 0; + } + size -= removed; + if (size > ARRAY_LIMIT / 2) { + return removed; + } + pairs = copyTablePairs(); + } else { + pairs = filterArray(predicate, previous); + if (size == previous) { + return 0; + } + } + retainPairs(pairs); + return previous - size; + } + + /** + * Copies dense survivors into the flat representation in encounter order. + * + * @return ordered key/value pairs of the current dense survivors + */ + private Object[] copyTablePairs() { + final Object[] pairs = new Object[size * 2]; + final Object[] compacted = pairs; + final int[] offset = { 0 }; + table().forEach((key, value) -> { + compacted[offset[0]++] = key; + compacted[offset[0]++] = value; + }); + return pairs; + } + + /** + * Compacts surviving flat pairs and clears the vacated references. + * + * @param predicate non-mutating predicate or captured predicate key, according to the operation + * @param previous live cardinality before filtering + * @return ordered surviving pairs, possibly in a smaller buffer + */ + private Object[] filterArray(final Predicate predicate, final int previous) { + Object[] pairs = (Object[]) entries; + int target = 0; + for (int i = 0; i < previous * 2; i += 2) { + if (predicate.test(value(pairs[i + 1]))) { + continue; + } + pairs[target++] = pairs[i]; + pairs[target++] = pairs[i + 1]; + } + size = target / 2; + if (size == previous) { + return pairs; + } + Arrays.fill(pairs, target, previous * 2, null); + if (size > 1 && size * 4 <= pairs.length) { + pairs = Arrays.copyOf(pairs, size * 2); + } + return pairs; + } + + /** + * Selects the empty, singleton or flat representation for an ordered array of survivors. + * + * @param pairs ordered key/value pairs for live survivors + */ + private void retainPairs(final Object[] pairs) { + entries = switch (size) { + case 0 -> { + singletonKey = null; + yield null; + } + case 1 -> { + singletonKey = (TermKey) pairs[0]; + yield pairs[1]; + } + default -> pairs; + }; + } + + /** + * Accumulates a value-derived count over live entries in encounter order. + * + * @param function non-mutating value-to-count function + * @return sum over live values + */ + long sum(final ToLongFunction function) { + if (size == 0) { + return 0; + } + if (size == 1) { + return function.applyAsLong(value(entries)); + } + long result = 0; + if (entries instanceof DenseTermTable) { + return table().sum(function); + } + final Object[] pairs = (Object[]) entries; + for (int i = 1; i < size * 2; i += 2) { + result += function.applyAsLong(value(pairs[i])); + } + return result; + } + + /** + * Visits every live entry in encounter order; the callback must not mutate this container. + * + * @param visitor non-mutating callback; true requests an early stop for predicate visitors + */ + void forEach(final BiConsumer visitor) { + if (size == 1) { + visitor.accept(singletonKey, value(entries)); + } else if (size > 1) { + if (entries instanceof DenseTermTable) { + table().forEach(visitor); + } else { + final Object[] pairs = (Object[]) entries; + for (int i = 0; i < size * 2; i += 2) { + visitor.accept((TermKey) pairs[i], value(pairs[i + 1])); + } + } + } + } + + /** + * Allocation-free positional traversal, only while the enclosing partition is retained. + * + * @param from inclusive starting offset of positional traversal + * @return the next live offset, or -1 at exhaustion + */ + + int nextIndex(final int from) { + if (entries instanceof DenseTermTable) { + return table().nextIndex(from); + } + return from < size ? from : -1; + } + + /** + * Reads the value at a live position supplied by positional traversal. + * + * @param index live positional offset returned by this container + * @return the non-null value at the supplied live position + */ + V valueAt(final int index) { + if (entries instanceof DenseTermTable) { + return table().valueAt(index); + } + if (size == 1) { + return value(entries); + } + return value(((Object[]) entries)[index * 2 + 1]); + } +} diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/DenseTermTable.java b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/DenseTermTable.java new file mode 100644 index 000000000..cec402699 --- /dev/null +++ b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/DenseTermTable.java @@ -0,0 +1,745 @@ +package fr.inria.corese.core.next.storage.impl.memory; + +import java.util.AbstractCollection; +import java.util.AbstractMap; +import java.util.AbstractSet; +import java.util.Arrays; +import java.util.Collection; +import java.util.Iterator; +import java.util.NoSuchElementException; +import java.util.Objects; +import java.util.Set; +import java.util.function.BiConsumer; +import java.util.function.Function; +import java.util.function.Predicate; +import java.util.function.ToLongFunction; + +/** + * Insertion-ordered term map backed by parallel entry arrays and a hash directory. + * + *

+ * {@code keys}, {@code data} and {@code hashes} have the same capacity. A live entry occupies the same offset in all + * three arrays; a null key marks a hole. {@code size} counts live entries and {@code used} bounds the occupied prefix, + * including holes. The power-of-two {@code positions} directory uses linear probing and stores entry offsets plus one, + * with zero denoting a vacant bucket. Cached full structural hashes reject unequal terms before RDF equality is tested. + * Keys and values must be non-null. + * + *

+ * Insertion appends to the ordered arrays. Removal repairs the probe cluster without moving surviving entries; periodic + * compaction packs entries stably and rebuilds the directory. Batch filtering performs one stable compaction. None of + * these operations changes the encounter order of surviving entries. + * + *

+ * Copy constructors clone every owned array while sharing keys and, unless a value mapper is supplied, values. The + * caller must copy mutable nested values. This class is not thread-safe: mutations require exclusive partition + * ownership under the database monitor. Iterators are not fail-fast and may only traverse an unchanged table, either + * under that monitor or through a retained snapshot. + * + *

+ * Equality and hashing retain the inherited Map entry semantics; capacities and holes are implementation state. + * + * @param non-null entry value type + */ +@SuppressWarnings("java:S2160") +final class DenseTermTable extends AbstractMap { + /** + * Maximum power-of-two directory capacity. + */ + private static final int MAX_BUCKETS = 1 << 30; + /** + * Parallel immutable term keys; null slots denote deleted entries. + */ + private TermKey[] keys; + /** + * Parallel non-null live values at the same offsets as the keys. + */ + private Object[] data; + /** + * Linear-probe directory storing live entry offsets plus one. + */ + private int[] positions; + /** + * Parallel cached structural hashes for filtering RDF equality checks. + */ + private int[] hashes; + /** + * Number of live entries or statements owned by this container. + */ + private int size; + /** + * Bound of the occupied entry prefix, including holes. + */ + private int used; + + /** + * Creates an empty dense table with the requested initial capacity. + */ + DenseTermTable() { + this(8); + } + + /** + * Creates an empty dense table with the requested initial capacity. + * + * @param capacity requested initial entry capacity + * @throws IllegalArgumentException if a capacity, term type or requested isolation level is unsupported + */ + DenseTermTable(int capacity) { + if (capacity < 0 || capacity > MAX_BUCKETS / 2) { + throw new IllegalArgumentException("Invalid capacity"); + } + capacity = Math.max(8, capacity); + keys = new TermKey[capacity]; + data = new Object[capacity]; + int buckets = 16; + while (capacity > (buckets >>> 1)) { + buckets <<= 1; + } + positions = new int[buckets]; + hashes = new int[capacity]; + } + + /** + * Copies buffers, capacities and holes without rehashing or reinserting entries. + * + * @param source source index or container; it must remain unchanged during copying or traversal + */ + + DenseTermTable(final DenseTermTable source) { + keys = source.keys.clone(); + data = source.data.clone(); + positions = source.positions.clone(); + hashes = source.hashes.clone(); + size = source.size; + used = source.used; + } + + /** + * Copy the ordered storage/directory, transforming only live values. + * + * @param source source index or container; it must remain unchanged during copying or traversal + * @param values mapper for live values in the copied, independently owned buffers + */ + + DenseTermTable(final DenseTermTable source, final Function values) { + this(source); + mapValues(values); + } + + /** + * Transform values in owned buffers without changing keys, holes or the directory. + * + * @param values mapper for live values in the copied, independently owned buffers + * @throws NullPointerException if an explicitly required argument or mapped value is null + */ + + void mapValues(final Function values) { + for (int i = 0; i < used; i++) { + if (keys[i] != null) { + data[i] = Objects.requireNonNull(values.apply(value(i))); + } + } + } + + /** + * Mixes a cached structural hash for the power-of-two probe directory. + * + * @param hash cached structural hash of the captured RDF value + * @return the mixed hash used to select a probe bucket + */ + private static int spread(final int hash) { + final int mixed = hash * 0x9E3779B9; + return mixed ^ (mixed >>> 16); + } + + /** + * Finds the probe bucket containing a key or the vacant bucket for its insertion. + * + * @param key search or insertion key; null denotes a wildcard only for pattern traversal + * @return the matching bucket or the vacant insertion bucket + */ + private int slot(final TermKey key) { + final int hash = key.hashCode(); + final int mask = positions.length - 1; + int slot = spread(hash) & mask; + + for (;;) { + final int index = positions[slot] - 1; + if (index < 0) { + return slot; + } + if (hashes[index] == hash && key.equals(keys[index])) { + return slot; + } + slot = (slot + 1) & mask; + } + } + + /** + * Reads an immutable RDF value or a value held in the container representation. + * + * @param index live positional offset returned by this container + * @return the stored RDF value or the non-null entry value at the specified position + */ + @SuppressWarnings("unchecked") + private V value(final int index) { + return (V) data[index]; + } + + /** + * {@inheritDoc} + * + * @return live entry count + */ + @Override + public int size() { + return size; + } + + /** + * {@inheritDoc} + * + * @param key search or insertion key; null denotes a wildcard only for pattern traversal + * @return the matching value, or null if absent + */ + @Override + public V get(final Object key) { + if (!(key instanceof TermKey term)) { + return null; + } + final int index = positions[slot(term)] - 1; + return index < 0 ? null : value(index); + } + + /** + * Search an already captured RDF value without constructing a transient TermKey. + * + * @param value non-null RDF value or entry value + * @param hash cached structural hash of the captured RDF value + * @return the matching value, or null if absent + */ + + V get(final fr.inria.corese.core.next.data.api.term.Value value, final int hash) { + final int mask = positions.length - 1; + int slot = spread(hash) & mask; + + for (;;) { + final int index = positions[slot] - 1; + if (index < 0) { + return null; + } + if (hashes[index] == hash && keys[index].matchesLookup(value, hash)) { + return value(index); + } + slot = (slot + 1) & mask; + } + } + + /** + * {@inheritDoc} + * + * @param key search or insertion key; null denotes a wildcard only for pattern traversal + * @param value non-null RDF value or entry value + * @return the existing value, or null after insertion + */ + @Override + public V putIfAbsent(final TermKey key, final V value) { + return insert(key, value, null, false); + } + + /** + * {@inheritDoc} + * + * @param key search or insertion key; null denotes a wildcard only for pattern traversal + * @param value non-null RDF value or entry value + * @return the previous value, or null for a new key + */ + @Override + public V put(final TermKey key, final V value) { + return insert(key, value, null, true); + } + + /** + * {@inheritDoc} + * + * @param key search or insertion key; null denotes a wildcard only for pattern traversal + * @param factory value factory invoked only for an absent key + * @return the existing or computed value, or null if computation returned null + */ + @Override + public V computeIfAbsent(final TermKey key, final Function factory) { + return insert(key, null, factory, false); + } + + /** + * Inserts an entry or returns its previous value using the requested replacement policy. + * + * @param key search or insertion key; null denotes a wildcard only for pattern traversal + * @param value non-null RDF value or entry value + * @param factory value factory invoked only for an absent key + * @param replace whether an existing value may be replaced + * @return the existing value, the computed absent value, or null after direct insertion + * @throws NullPointerException if an explicitly required argument or mapped value is null + */ + private V insert(final TermKey key, final V value, final Function factory, + final boolean replace) { + int slot = slot(key); + final int index = positions[slot] - 1; + + if (index >= 0) { + final V previous = value(index); + if (replace) { + data[index] = Objects.requireNonNull(value); + } + return previous; + } + final V inserted = factory == null ? Objects.requireNonNull(value) : factory.apply(key); + if (inserted == null) { + return null; + } + if (prepareInsertionSpace()) { + slot = slot(key); + } + keys[used] = key; + data[used] = inserted; + hashes[used] = key.hashCode(); + positions[slot] = ++used; + size++; + return factory == null ? null : inserted; + } + + /** + * Reserves entry and directory capacity without changing surviving entry order. + * + * @return true if entry positions were compacted or the directory was rebuilt + * @throws OutOfMemoryError if the maximum supported directory capacity is exhausted + */ + private boolean prepareInsertionSpace() { + boolean reindexed = false; + if (used == keys.length) { + if (used - size >= Math.max(1, size >>> 2)) { + compact(); + reindexed = true; + } else { + final int capacity = Math.addExact(keys.length, Math.max(1, keys.length >>> 1)); + keys = Arrays.copyOf(keys, capacity); + data = Arrays.copyOf(data, capacity); + hashes = Arrays.copyOf(hashes, capacity); + } + } + if (size + 1 > (positions.length >>> 1)) { + if (positions.length == MAX_BUCKETS) { + throw new OutOfMemoryError("Term table capacity exhausted"); + } + rehash(positions.length << 1); + reindexed = true; + } + return reindexed; + } + + /** + * {@inheritDoc} + * + * @param key search or insertion key; null denotes a wildcard only for pattern traversal + * @param replacement non-null replacement for an existing value + * @return the previous value, or null if the key was absent + * @throws NullPointerException if an explicitly required argument or mapped value is null + */ + @Override + public V replace(final TermKey key, final V replacement) { + final int index = positions[slot(key)] - 1; + if (index < 0) { + return null; + } + final V previous = value(index); + data[index] = Objects.requireNonNull(replacement); + return previous; + } + + /** + * {@inheritDoc} + * + * @param key search or insertion key; null denotes a wildcard only for pattern traversal + * @return number of matching statements removed + */ + @Override + public V remove(final Object key) { + if (!(key instanceof TermKey term)) { + return null; + } + final int hole = slot(term); + final int index = positions[hole] - 1; + + if (index < 0) { + return null; + } + final V previous = value(index); + keys[index] = null; + data[index] = null; + size--; + closeProbeGap(hole); + compactAfterRemoval(); + return previous; + } + + /** + * Repairs the linear-probe cluster after a bucket is removed. + * + * @param hole vacant bucket whose probe cluster must be repaired + */ + private void closeProbeGap(int hole) { + final int mask = positions.length - 1; + for (int next = (hole + 1) & mask; positions[next] != 0; next = (next + 1) & mask) { + final int home = spread(hashes[positions[next] - 1]) & mask; + if (((next - home) & mask) >= ((next - hole) & mask)) { + positions[hole] = positions[next]; + hole = next; + } + } + positions[hole] = 0; + } + + /** + * Applies the existing compaction and buffer-shrink thresholds after a single removal. + */ + private void compactAfterRemoval() { + if (size == 0) { + used = 0; + if (keys.length > 16) { + resetBuffers(); + } + } else if (used >= 64 && size <= used / 2) { + compact(); + if (size <= keys.length / 4) { + shrinkEntryBuffers(); + } + } + } + + /** + * Replaces oversized empty buffers with the minimum entry and directory capacities. + */ + private void resetBuffers() { + keys = new TermKey[8]; + data = new Object[8]; + hashes = new int[8]; + positions = new int[16]; + } + + /** + * Shrinks parallel entry arrays together; live offsets and the directory remain valid. + */ + private void shrinkEntryBuffers() { + final int capacity = Math.max(8, size * 2); + keys = Arrays.copyOf(keys, capacity); + data = Arrays.copyOf(data, capacity); + hashes = Arrays.copyOf(hashes, capacity); + } + + /** + * Packs live parallel entries stably and rebuilds the probe directory. + */ + private void compact() { + // Stable packing bounds hole scans. Reuse the directory during churn. + int target = 0; + for (int i = 0; i < used; i++) { + if (keys[i] != null) { + keys[target] = keys[i]; + hashes[target] = hashes[i]; + data[target++] = data[i]; + } + } + Arrays.fill(keys, target, used, null); + Arrays.fill(data, target, used, null); + used = target; + Arrays.fill(positions, 0); + fillPositions(); + } + + /** + * Stable batch filtering. The predicate must not access or mutate this table. + * + * @param predicate non-mutating predicate or captured predicate key, according to the operation + * @return number of removed entries + */ + + int removeIf(final Predicate predicate) { + final int previous = size; + for (int i = 0; i < used; i++) { + if (keys[i] != null && predicate.test(value(i))) { + keys[i] = null; + data[i] = null; + size--; + } + } + if (size == previous) { + return 0; + } + compact(); + if (size == 0 && keys.length > 16) { + resetBuffers(); + } else if (size > 0 && size <= keys.length / 4) { + shrinkEntryBuffers(); + } + return previous - size; + } + + /** + * Replaces the probe directory and indexes all live entry positions. + * + * @param capacity requested initial entry capacity + */ + private void rehash(final int capacity) { + positions = new int[capacity]; + fillPositions(); + } + + /** + * Indexes live entry offsets in the current empty probe directory. + */ + private void fillPositions() { + final int mask = positions.length - 1; + for (int i = 0; i < used; i++) { + if (keys[i] != null) { + int slot = spread(hashes[i]) & mask; + while (positions[slot] != 0) { + slot = (slot + 1) & mask; + } + positions[slot] = i + 1; + } + } + } + + /** + * Traverses matching statements or live values until the visitor requests an early stop. + * + * @param visitor non-mutating callback; true requests an early stop for predicate visitors + * @return true when a matching value caused the visitor to request an early stop + */ + boolean visit(final Predicate visitor) { + for (int i = 0; i < used; i++) { + if (keys[i] != null && visitor.test(value(i))) { + return true; + } + } + return false; + } + + /** + * Positions in an immutable, retained partition; holes keep their encounter order. + * + * @param from inclusive starting offset of positional traversal + * @return the next live offset, or -1 at exhaustion + */ + + int nextIndex(int from) { + while (from < used && keys[from] == null) { + from++; + } + return from < used ? from : -1; + } + + /** + * Reads the value at a live position supplied by positional traversal. + * + * @param index live positional offset returned by this container + * @return the non-null value at the supplied live position + */ + V valueAt(final int index) { + return value(index); + } + + /** + * Accumulates a value-derived count over live entries in encounter order. + * + * @param function non-mutating value-to-count function + * @return sum over live values + */ + long sum(final ToLongFunction function) { + long total = 0; + for (int i = 0; i < used; i++) { + if (keys[i] != null) { + total += function.applyAsLong(value(i)); + } + } + return total; + } + + /** + * {@inheritDoc} + * + * @param visitor non-mutating callback; true requests an early stop for predicate visitors + */ + @Override + public void forEach(final BiConsumer visitor) { + for (int i = 0; i < used; i++) { + if (keys[i] != null) { + visitor.accept(keys[i], value(i)); + } + } + } + + /** + * Ordered traversal over an unchanged table; it must not overlap mutation of its buffers. + * + * @param ordered traversal item type + */ + private abstract class OrderedIterator implements Iterator { + + /** Creates an ordered iterator beginning at the first entry offset. */ + private OrderedIterator() { + } + + /** + * Next ordered entry offset to inspect. + */ + private int next; + + /** + * Reports whether this unchanged ordered traversal has another live item. + * + * @return true when another live entry remains + */ + public boolean hasNext() { + while (next < used && keys[next] == null) { + next++; + } + return next < used; + } + + /** + * Returns the next match in this sequential traversal. + * + * @return the next item in encounter order + * @throws NoSuchElementException if this ordered traversal is exhausted + */ + public T next() { + if (!hasNext()) { + throw new NoSuchElementException(); + } + return item(next++); + } + + /** + * Reads the item at the live offset selected by this iterator. + * + * @param index live positional offset returned by this container + * @return the item at the supplied live offset + */ + abstract T item(final int index); + } + + /** + * {@inheritDoc} + * + * @return an insertion-ordered view of live map entries + */ + @Override + public Set> entrySet() { + return new AbstractSet<>() { + /** + * Returns the number of live entries. + * + * @return live entry count + */ + public int size() { + return DenseTermTable.this.size; + } + + /** + * Creates an ordered iterator over this unchanged table view. + * + * @return an insertion-ordered iterator of the unchanged view + */ + public Iterator> iterator() { + return new OrderedIterator<>() { + /** + * Reads the item at the live offset selected by this iterator. + * + * @param i live positional offset supplied by traversal + * @return the item at the supplied live offset + */ + Entry item(final int i) { + return new SimpleImmutableEntry<>(keys[i], value(i)); + } + }; + } + }; + } + + /** + * {@inheritDoc} + * + * @return an insertion-ordered view of live values + */ + @Override + public Collection values() { + return new AbstractCollection<>() { + /** + * Returns the number of live entries. + * + * @return live entry count + */ + public int size() { + return DenseTermTable.this.size; + } + + /** + * Creates an ordered iterator over this unchanged table view. + * + * @return an insertion-ordered iterator of the unchanged view + */ + public Iterator iterator() { + return new OrderedIterator<>() { + /** + * Reads the item at the live offset selected by this iterator. + * + * @param i live positional offset supplied by traversal + * @return the item at the supplied live offset + */ + V item(final int i) { + return value(i); + } + }; + } + }; + } + + /** + * {@inheritDoc} + * + * @return an insertion-ordered view of live term keys + */ + @Override + public Set keySet() { + return new AbstractSet<>() { + /** + * Returns the number of live entries. + * + * @return live entry count + */ + public int size() { + return DenseTermTable.this.size; + } + + /** + * Creates an ordered iterator over this unchanged table view. + * + * @return an insertion-ordered iterator of the unchanged view + */ + public Iterator iterator() { + return new OrderedIterator<>() { + /** + * Reads the item at the live offset selected by this iterator. + * + * @param i live positional offset supplied by traversal + * @return the item at the supplied live offset + */ + TermKey item(final int i) { + return keys[i]; + } + }; + } + }; + } +} diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/InMemoryStatementStore.java b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/InMemoryStatementStore.java deleted file mode 100644 index 2d15b4097..000000000 --- a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/InMemoryStatementStore.java +++ /dev/null @@ -1,338 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.memory; - -import fr.inria.corese.core.next.data.api.term.IRI; -import fr.inria.corese.core.next.data.api.term.Resource; -import fr.inria.corese.core.next.data.api.model.Statement; -import fr.inria.corese.core.next.data.api.term.Value; - -import java.util.Arrays; -import java.util.Objects; -import java.util.HashSet; -import java.util.Set; - -import java.util.stream.Collectors; - -/** - * In-memory statement and graph store with optimistic, isolated transaction snapshots. - * - *

The transaction state is bound to the current thread. A transaction must - * therefore be started, used, and completed by the same thread (the normal - * repository-connection contract). Applications that transfer work between - * executor threads must keep the connection and transaction on one worker, or - * provide a higher-level transaction context.

- * - *

Beginning a transaction copies the committed statement and graph sets. - * This gives predictable snapshot isolation, but its memory cost is - * proportional to the dataset size. This backend is intended for tests and - * small to medium in-memory datasets; large persistent datasets should use a - * storage engine with native MVCC.

- */ -final class InMemoryStatementStore { - - private final Database database; - - private static final class Database { - private State committed = new State(new HashSet<>(), new HashSet<>()); - private long version; - } - private final ThreadLocal transaction = new ThreadLocal<>(); - - private record State(Set statements, Set graphs) { } - - private State state() { - State local = transaction.get(); - return local == null ? database.committed : local; - } - - private void changed(boolean changed) { - if (changed && transaction.get() == null) { - database.version++; - } - } - - /** - * Begins a transaction by snapshotting committed statements and graphs. - * - * @return current database version for optimistic concurrency checks - * @throws IllegalStateException if a transaction is already active on this thread - */ - long begin() { - synchronized (database) { - if (transaction.get() != null) { - throw new IllegalStateException("A transaction is already active on this thread"); - } - transaction.set(new State(new HashSet<>(database.committed.statements()), new HashSet<>(database.committed.graphs()))); - return database.version; - } - } - - /** - * Commits the active transaction to shared database state if no conflict occurred. - * - * @param expectedVersion the database version captured at begin - * @throws IllegalStateException if concurrent modifications occurred or no transaction is active - */ - void commit(long expectedVersion) { - synchronized (database) { - if (database.version != expectedVersion) { - throw new IllegalStateException("Concurrent storage modification; transaction must be rolled back"); - } - database.committed = Objects.requireNonNull(transaction.get(), "No active transaction"); - transaction.remove(); - database.version++; - } - } - - /** - * Rolls back and discards the active transaction on this thread. - */ - void rollback() { - transaction.remove(); - } - - /** - * Creates an empty named graph in the store. - * - * @param graph the named graph identifier (must not be null) - * @return true if the graph was created, false if it already existed - * @throws NullPointerException if graph is null - */ - public boolean createGraph(Resource graph) { - synchronized (database) { - boolean added = state().graphs().add(Objects.requireNonNull(graph, "graph")); - changed(added); - return added; - } - } - - /** - * Drops a named graph and all statements contained in it. - * - * @param graph the named graph identifier (must not be null) - * @return true if the graph was removed, false if it did not exist - * @throws NullPointerException if graph is null - */ - public boolean dropGraph(Resource graph) { - synchronized (database) { - clearContext(Objects.requireNonNull(graph, "graph")); - boolean removed = state().graphs().remove(graph); - changed(removed); - return removed; - } - } - - /** - * Constructs a new InMemoryStatementStore with an empty statement set. - */ - public InMemoryStatementStore() { - this(new Database()); - } - - /** - * Constructs a statement store sharing the given database storage. - * - * @param database the shared in-memory database - */ - private InMemoryStatementStore(Database database) { - this.database = database; - } - - /** - * Opens a new session sharing the committed state of this store. - * - * @return a new session-scoped {@link InMemoryStatementStore} - */ - InMemoryStatementStore openSession() { - return new InMemoryStatementStore(database); - } - - /** - * Adds a statement to the in-memory store. - * - * @param stmt the statement to add (must not be null) - * @return true if the statement was added, false if it already existed - */ - public boolean add(Statement stmt) { - synchronized (database) { - boolean graphAdded = stmt.getContext() != null && state().graphs().add(stmt.getContext()); - boolean added = state().statements().add(stmt); - changed(added || graphAdded); - return added; - } - } - - /** - * Removes a statement from the in-memory store. - * - * @param stmt the statement to remove (must not be null) - * @return true if the statement was removed, false if it did not exist - */ - public boolean remove(Statement stmt) { - synchronized (database) { - boolean removed = state().statements().remove(stmt); - changed(removed); - return removed; - } - } - - /** - * Checks if a statement exists in the in-memory store. - * - * @param stmt the statement to check (must not be null) - * @return true if the statement exists - */ - public boolean contains(Statement stmt) { - synchronized (database) { - return state().statements().contains(stmt); - } - } - - /** - * Finds all statements matching the given pattern. - * - * @param s subject filter (null = any) - * @param p predicate filter (null = any) - * @param o object filter (null = any) - * @param contexts context filters (null/empty = any) - * @return set of matching statements (never null, may be empty) - */ - public Set find(Resource s, IRI p, Value o, Resource[] contexts) { - synchronized (database) { - return state().statements().stream() - .filter(stmt -> matches(stmt, s, p, o, contexts)) - .collect(Collectors.toSet()); - } - } - - /** - * Returns the total number of statements in the store. - * - * @return statement count - */ - public int size() { - synchronized (database) { - return state().statements().size(); - } - } - - /** - * Removes all statements from the store. - */ - public void clear() { - synchronized (database) { - changed(!state().statements().isEmpty()); - state().statements().clear(); - } - } - - /** - * Removes all statements from a specific context (named graph). - * - * @param context the context to clear (must not be null) - */ - public void clearContext(Resource context) { - synchronized (database) { - changed(state().statements().removeIf(stmt -> context.equals(stmt.getContext()))); - } - } - - /** - * Returns all unique subject resources in the store. - * - * @return set of all subjects (never null) - */ - public Set getSubjects() { - synchronized (database) { - return state().statements().stream() - .map(Statement::getSubject) - .collect(Collectors.toSet()); - } - } - - /** - * Returns all unique predicate IRIs in the store. - * - * @return set of all predicates (never null) - */ - public Set getPredicates() { - synchronized (database) { - return state().statements().stream() - .map(Statement::getPredicate) - .collect(Collectors.toSet()); - } - } - - /** - * Returns all unique object values in the store. - * - * @return set of all objects (never null) - */ - public Set getObjects() { - synchronized (database) { - return state().statements().stream() - .map(Statement::getObject) - .collect(Collectors.toSet()); - } - } - - /** - * Returns all unique context identifiers in the store. - * - * @return set of all contexts (never null, excludes null contexts) - */ - public Set getContexts() { - synchronized (database) { - return Set.copyOf(state().graphs()); - } - } - - /** - * Checks if a statement matches the given pattern. - * - * @param stmt the statement to test - * @param s subject filter (null = wildcard) - * @param p predicate filter (null = wildcard) - * @param o object filter (null = wildcard) - * @param contexts context filters (null/empty = wildcard) - * @return true if the statement matches all non-null filters - */ - private boolean matches(Statement stmt, Resource s, IRI p, Value o, Resource[] contexts) { - if (s != null && !s.equals(stmt.getSubject())) { - return false; - } - - if (p != null && !p.equals(stmt.getPredicate())) { - return false; - } - - if (o != null && !o.equals(stmt.getObject())) { - return false; - } - - if (contexts != null && contexts.length > 0) { - return matchesContext(stmt.getContext(), contexts); - } - - return true; - } - - /** - * Determines whether a statement's context matches any of the provided context filters. - * - *

The comparison relies on {@link Objects#equals(Object, Object)}, ensuring - * proper semantic equality between RDF {@link Resource} instances. In particular, - * this preserves type distinctions (e.g., IRI vs blank node) and correctly handles - * {@code null} values, which represent the default graph.

- * - * @param stmtContext the context of the statement, or {@code null} if it belongs - * to the default graph - * @param contexts the context filters to match against; must not be {@code null} - * or empty - * @return {@code true} if the statement's context is equal to at least one of the - * provided contexts, {@code false} otherwise - */ - private boolean matchesContext(Resource stmtContext, Resource[] contexts) { - return Arrays.stream(contexts) - .anyMatch(ctx -> Objects.equals(ctx, stmtContext)); - } -} diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/IndexedStatementStore.java b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/IndexedStatementStore.java new file mode 100644 index 000000000..1719dae35 --- /dev/null +++ b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/IndexedStatementStore.java @@ -0,0 +1,1805 @@ +package fr.inria.corese.core.next.storage.impl.memory; + +import fr.inria.corese.core.next.data.Values; +import fr.inria.corese.core.next.data.api.model.SimpleStatement; +import fr.inria.corese.core.next.data.api.model.Statement; +import fr.inria.corese.core.next.data.api.term.BNode; +import fr.inria.corese.core.next.data.api.term.IRI; +import fr.inria.corese.core.next.data.api.term.Literal; +import fr.inria.corese.core.next.data.api.term.Resource; +import fr.inria.corese.core.next.data.api.term.SimpleBNode; +import fr.inria.corese.core.next.data.api.term.SimpleDate; +import fr.inria.corese.core.next.data.api.term.SimpleDateTime; +import fr.inria.corese.core.next.data.api.term.SimpleDecimal; +import fr.inria.corese.core.next.data.api.term.SimpleDouble; +import fr.inria.corese.core.next.data.api.term.SimpleDuration; +import fr.inria.corese.core.next.data.api.term.SimpleIRI; +import fr.inria.corese.core.next.data.api.term.SimpleInteger; +import fr.inria.corese.core.next.data.api.term.SimpleLiteral; +import fr.inria.corese.core.next.data.api.term.SimpleTime; +import fr.inria.corese.core.next.data.api.term.SimpleTriple; +import fr.inria.corese.core.next.data.api.term.Triple; +import fr.inria.corese.core.next.data.api.term.Value; +import fr.inria.corese.core.next.storage.api.config.StorageConfig; +import fr.inria.corese.core.next.storage.api.exception.ErrorCode; +import fr.inria.corese.core.next.storage.api.exception.StorageException; +import fr.inria.corese.core.next.storage.api.lifecycle.LifecycleState; +import fr.inria.corese.core.next.storage.api.lifecycle.StorageLifecycle; +import fr.inria.corese.core.next.storage.api.model.StatementPattern; +import fr.inria.corese.core.next.storage.api.model.StorageStatistics; +import fr.inria.corese.core.next.storage.api.operations.MetadataOperations; +import fr.inria.corese.core.next.storage.api.operations.MutationOperations; +import fr.inria.corese.core.next.storage.api.operations.QueryOperations; +import fr.inria.corese.core.next.storage.api.operations.StatementSpliterator; +import fr.inria.corese.core.next.storage.api.transaction.IsolationLevel; +import fr.inria.corese.core.next.storage.api.transaction.Transaction; +import fr.inria.corese.core.next.storage.api.transaction.TransactionManager; +import fr.inria.corese.core.next.storage.api.transaction.TransactionState; + +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.Comparator; +import java.util.Iterator; +import java.util.LinkedHashMap; +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Map; +import java.util.Objects; +import java.util.Optional; +import java.util.Set; +import java.util.Spliterator; +import java.util.UUID; +import java.util.function.Consumer; +import java.util.function.Function; +import java.util.function.Predicate; +import java.util.stream.Stream; +import java.util.stream.StreamSupport; + +/** + * Session-scoped operations over a shared, partitioned in-memory database. + * + *

+ * The database monitor protects publication, transactions and partition ownership counts. Each live {@code State} owns + * one reference to each partition in its graph registry. A transaction or lazy read captures its own state and + * increments those references. Before mutating a shared partition, an owner copies all three index containers and + * transfers its reference to the private copy. Immutable statements and term keys remain shared. CLEAR replaces a + * partition with an empty partition; old readers retain the original version. Graph ordinals preserve registry order, + * and empty named graphs remain visible until DROP. + * + *

+ * A lazy read owns its captured state independently of the session or transaction that created it. Traversal and + * consumer callbacks run outside the database monitor over unchanged partitions. Releasing a cursor or transaction + * decrements references under the monitor; the last owner allows the old containers to be reclaimed. Stream consumers + * must close their streams, including on early exit or exception. + * + *

+ * Batch ingestion stages a single pass over the source before atomically publishing completed partitions. A failed + * batch leaves the visible state unchanged. Dirty transactions publish only if the committed database version still + * matches the version captured at begin; a conflict leaves rollback available to the caller. + */ +final class IndexedStatementStore + implements QueryOperations, MutationOperations, MetadataOperations, TransactionManager { + // One batch-wide reservation: at most 65,536 HashMap buckets, even for many graphs. + /** + * Upper bound on the batch-wide initial canonicalization reservation. + */ + private static final int MAX_INITIAL_STAGING_TERMS = 49_152; + /** + * Shared lifecycle monitor and committed database state. + */ + private final Database database; + /** + * Active transaction owned by this session, or null. + */ + private SnapshotTransaction transaction; + /** + * Whether this session has relinquished its operation lifecycle. + */ + private boolean closed; + /** + * Whether this session has an atomic batch in progress. + */ + private boolean loading; + + /** One graph and its three permutations; references count State owners, not rows. */ + private static final class Partition { + // Number of live States owning this partition, guarded by the database monitor. + /** + * Number of live State owners, guarded by the database monitor. + */ + private int references = 1; + /** + * Stable graph registry position. + */ + private final long ordinal; + /** + * Canonical graph identifier; null denotes the default graph. + */ + private final Resource context; + /** + * Subject/predicate/object permutation of this graph. + */ + private final TripleIndex spo; + /** + * Predicate/object/subject permutation of this graph. + */ + private final TripleIndex pos; + /** + * Object/subject/predicate permutation of this graph. + */ + private final TripleIndex osp; + + /** + * Creates a graph partition owning the supplied permutations or empty indexes. + * + * @param context canonical graph identifier; null denotes the default graph + * @param ordinal stable graph registry ordinal + */ + private Partition(final Resource context, final long ordinal) { + this(context, ordinal, new TripleIndex(TripleIndex.Order.SPO), new TripleIndex(TripleIndex.Order.POS), + new TripleIndex(TripleIndex.Order.OSP)); + } + + /** + * Creates a graph partition owning the supplied permutations or empty indexes. + * + * @param context canonical graph identifier; null denotes the default graph + * @param ordinal stable graph registry ordinal + * @param spo SPO index owning immutable statements and canonical keys + * @param pos POS permutation of the same graph partition + * @param osp OSP permutation of the same graph partition + */ + private Partition(final Resource context, final long ordinal, final TripleIndex spo, final TripleIndex pos, + final TripleIndex osp) { + this.context = context; + this.ordinal = ordinal; + this.spo = spo; + this.pos = pos; + this.osp = osp; + } + + /** + * Constructs a complete partition and releases its SPO staging dictionary before publication. + * + * @param spo SPO index owning immutable statements and canonical keys + * @param context canonical graph identifier; null denotes the default graph + * @param ordinal stable graph registry ordinal + * @return a complete graph partition with no retained staging dictionary + */ + private static Partition build(final TripleIndex spo, final Resource context, final long ordinal) { + final TripleIndex.SecondaryIndexes secondary = TripleIndex.buildSecondaryIndexes(spo); + final Partition partition = new Partition(context, ordinal, spo, secondary.pos(), secondary.osp()); + spo.finishStaging(); + return partition; + } + + /** + * Copies owned mutable containers while sharing immutable keys and statements. + * + * @return an independently owned copy of the mutable containers + */ + private Partition copy() { + return new Partition(context, ordinal, spo.copy(), pos.copy(), osp.copy()); + } + + /** + * Adds a statement or keyed value if it is absent. + * + * @param s captured statement to index + * @return true when an absent entry was added, false when it was already present + */ + private boolean add(Statement s) { + final TermKey subject = key(s.getSubject(), null, null, spo, pos, osp); + final TermKey predicate = key(s.getPredicate(), subject, null, pos, spo, osp); + final TermKey object = key(s.getObject(), subject, predicate, osp, spo, pos); + s = sharedStatement(s, subject, predicate, object, context); + if (!spo.add(s, subject, predicate, object, osp)) { + return false; + } + pos.add(s, subject, predicate, object, spo); + osp.add(s, subject, predicate, object, pos); + return true; + } + + /** + * Reuses a canonical RDF term key across roles, creating one only for a previously unseen term. + * + * @param value non-null RDF value or entry value + * @param subject canonical subject key, or null for a wildcard search + * @param predicate canonical predicate key; null denotes a wildcard for search operations + * @param primary preferred role-root index for canonical lookup + * @param secondary second role-root index for canonical lookup + * @param tertiary third role-root index for canonical lookup + * @return a reusable canonical key, or a newly captured key for an unseen RDF term + */ + private TermKey key(final Value value, final TermKey subject, final TermKey predicate, + final TripleIndex primary, final TripleIndex secondary, final TripleIndex tertiary) { + final int hash = TermKey.structuralHash(value); + if (subject != null && subject.matchesLookup(value, hash)) { + return subject; + } + if (predicate != null && predicate.matchesLookup(value, hash)) { + return predicate; + } + // Role roots retain canonical keys; absent RDF terms require a new key. + TermKey canonical = primary.existingKey(value, hash); + if (canonical == null) { + canonical = secondary.existingKey(value, hash); + } + if (canonical == null) { + canonical = tertiary.existingKey(value, hash); + } + return canonical == null ? new TermKey(value, hash) : canonical; + } + + /** + * Removes matching content while preserving the order of survivors. + * + * @param s captured statement to index + * @return true when an entry was removed, false when it was absent + */ + private boolean remove(final Statement s) { + final TermKey subject = new TermKey(s.getSubject()); + final TermKey predicate = new TermKey(s.getPredicate()); + final TermKey object = new TermKey(s.getObject()); + if (!spo.remove(subject, predicate, object)) { + return false; + } + pos.remove(subject, predicate, object); + osp.remove(subject, predicate, object); + return true; + } + + /** + * Removes selected index content with stable filtering and unchanged compaction rules. + * + * @param pattern captured search pattern + */ + private void removeMatches(final StatementPattern pattern) { + final TermKey subject = TermKey.nullable(pattern.getSubject()); + final TermKey predicate = TermKey.nullable(pattern.getPredicate()); + final TermKey object = TermKey.nullable(pattern.getObject()); + spo.removeMatches(subject, predicate, object); + pos.removeMatches(subject, predicate, object); + osp.removeMatches(subject, predicate, object); + } + + /** + * Selects the permutation appropriate for the bound prefix of a search pattern. + * + * @param p captured search pattern + * @return the SPO, POS or OSP permutation selected for this pattern + */ + private TripleIndex index(final StatementPattern p) { + if (p.getSubject() != null && (p.getPredicate() != null || p.getObject() == null)) { + return spo; + } + if (p.getPredicate() != null) { + return pos; + } + return p.getObject() != null ? osp : spo; + } + + /** + * Reports whether the session-visible or retained graph content contains a match. + * + * @param statement statement whose RDF identities have been captured + * @return true when at least one matching statement exists + */ + private boolean contains(final Statement statement) { + return spo.count( + StatementPattern.of(statement.getSubject(), statement.getPredicate(), statement.getObject())) != 0; + } + } + + /** Ordered graph registry whose partition references are acquired and released together. */ + private static final class State { + + /** Creates an empty ordered registry with exclusive ownership of its map. */ + private State() { + } + + // A null key is the default partition. Empty named partitions are retained until DROP. + /** + * Ordered partition registry; empty named graphs are retained until DROP. + */ + private final Map graphs = new LinkedHashMap<>(); + /** + * Number of live entries or statements owned by this container. + */ + private long size; + /** + * Ordinal assigned to the next created graph. + */ + private long nextOrdinal; + + /** + * Copies owned mutable containers while sharing immutable keys and statements. + * + * @return an independently owned copy of the mutable containers + */ + private State copy() { + final State copy = new State(); + graphs.forEach((graph, partition) -> { + partition.references++; + copy.graphs.put(graph, partition); + }); + copy.size = size; + copy.nextOrdinal = nextOrdinal; + return copy; + } + + /** + * Capture selected graphs only; writes to unrelated graphs need no cursor-induced COW. + * + * @param contexts selected graph identifiers; an empty array selects all graphs + * @return an independently owned copy of the mutable containers + */ + private State copy(final Resource[] contexts) { + if (contexts.length == 0) return copy(); + final State copy = new State(); + final List selected = new ArrayList<>(); + for (final Resource graph : contexts) { + final Partition partition = graphs.get(graph); + if (partition != null) selected.add(partition); + } + selected.sort(Comparator.comparingLong(partition -> partition.ordinal)); + for (final Partition partition : selected) { + partition.references++; + copy.graphs.put(partition.context, partition); + copy.size += partition.spo.size(); + } + copy.nextOrdinal = nextOrdinal; + return copy; + } + + /** + * Detaches only a shared partition, preserving existing readers and graph order. + * + * @param graph named graph identifier or staging registry key, according to the operation + * @return the exclusively owned partition, or null if the graph is absent + */ + private Partition writable(final Resource graph) { + final Partition partition = graphs.get(graph); + if (partition != null && partition.references > 1) { + final Partition replacement = partition.copy(); + partition.references--; + graphs.put(graph, replacement); + return replacement; + } + return partition; + } + + /** + * Replaces a partition with an empty version, preserving graph identity and registry position. + * + * @param partition partition retained or exclusively owned by the supplied state + */ + private void empty(final Partition partition) { + // Keep the graph's identity and registry position. Readers continue to own + // the old partition; clearing never needs to clone its three indexes. + final Partition replacement = new Partition(partition.context, partition.ordinal); + graphs.put(partition.context, replacement); + partition.references--; + } + + /** Releases this state's ownership; clearing the registry makes repeated release harmless. */ + private void release() { + graphs.values().forEach(partition -> partition.references--); + graphs.clear(); + } + } + + /** + * Shared database state and lifecycle monitor for all sessions. + */ + private static final class Database implements StorageLifecycle { + + /** Creates an uninitialized database and its exclusively owned empty committed state. */ + private Database() { + } + + /** + * Committed state whose partition references are owned by the database. + */ + private State committed = new State(); + /** + * Committed version captured for optimistic conflict detection. + */ + private long version; + /** + * Current shared database lifecycle status. + */ + private LifecycleState lifecycleState = LifecycleState.NOT_INITIALIZED; + /** + * Configuration available during the running lifecycle. + */ + private StorageConfig config; + + /** + * {@inheritDoc} + * + * @param config storage configuration selecting this backend + * @throws IllegalStateException if the required representation, ownership or lifecycle state is not satisfied + * @throws NullPointerException if an explicitly required argument or mapped value is null + */ + @Override + public synchronized void initialize(final StorageConfig config) { + Objects.requireNonNull(config, "config"); + if (lifecycleState != LifecycleState.NOT_INITIALIZED) { + throw new IllegalStateException("Already initialized"); + } + this.config = config; + lifecycleState = LifecycleState.RUNNING; + } + + /** + * {@inheritDoc} + * + * @throws IllegalStateException if the required representation, ownership or lifecycle state is not satisfied + */ + @Override + public synchronized void shutdown() { + if (lifecycleState != LifecycleState.RUNNING) { + throw new IllegalStateException("Storage is not running"); + } + committed.release(); + committed = new State(); + config = null; + lifecycleState = LifecycleState.SHUTDOWN; + } + + /** + * {@inheritDoc} + * + * @return the current lifecycle or transaction status + */ + @Override + public synchronized LifecycleState getState() { + return lifecycleState; + } + + /** + * {@inheritDoc} + * + * @return current configuration when the database is running + */ + @Override + public synchronized Optional getConfig() { + return Optional.ofNullable(config); + } + } + + /** + * Creates session operations sharing the supplied database, or a new uninitialized database. + */ + IndexedStatementStore() { + this(new Database()); + } + + /** + * Creates session operations sharing the supplied database, or a new uninitialized database. + * + * @param database shared database monitor and state + */ + private IndexedStatementStore(final Database database) { + this.database = database; + } + + /** + * Returns the shared database lifecycle controller. + * + * @return the shared database lifecycle controller + */ + StorageLifecycle lifecycle() { + return database; + } + + /** + * Reports whether this session and the shared database accept operations. + * + * @return true when the session is open and the database is running + */ + boolean isOpen() { + synchronized (database) { + return !closed && database.lifecycleState == LifecycleState.RUNNING; + } + } + + /** + * Creates a session sharing the database and owning its own transaction lifecycle. + * + * @return a new session sharing the database + * @throws IllegalStateException if the session is closed or its transaction belongs to another thread + */ + IndexedStatementStore openSession() { + synchronized (database) { + checkAccess(); + return new IndexedStatementStore(database); + } + } + + /** + * Closes this session, rolling back its active transaction; independent read snapshots remain retained. + * + * @throws IllegalStateException if the required representation, ownership or lifecycle state is not satisfied + */ + void closeSession() { + synchronized (database) { + if (closed) { + return; + } + if (loading) { + throw new IllegalStateException("A batch is active on this session"); + } + if (transaction != null) { + transaction.rollback(); + } + closed = true; + } + } + + /** + * Requires an open session and, when active, access from its transaction owner thread. + * + * @throws IllegalStateException if the required representation, ownership or lifecycle state is not satisfied + */ + private void checkAccess() { + if (!isOpen()) { + throw new IllegalStateException("Storage session is closed"); + } + if (transaction != null) { + transaction.checkOwner(); + } + } + + /** + * Requires session access and excludes overlapping batch mutations on this session. + * + * @throws IllegalStateException if the required representation, ownership or lifecycle state is not satisfied + */ + private void checkWrite() { + checkAccess(); + if (loading) { + throw new IllegalStateException("A batch is active on this session"); + } + } + + /** + * Returns the session-visible state; the database monitor must be held. + * + * @return the transaction state, or the committed state outside a transaction + */ + private State state() { + return transaction == null ? database.committed : transaction.state; + } + + /** + * Marks a successful mutation in the current transaction or increments the committed version. + * + * @param changed whether this mutation changed visible content + */ + private void changed(final boolean changed) { + if (!changed) { + return; + } + if (transaction == null) { + database.version++; + } else { + transaction.dirty = true; + } + } + + /** + * Captures RDF identities without changing the statement or pattern semantics. + * + * @param value non-null RDF value or entry value + * @return the original supported native value or a captured representation of its RDF identity + * @throws IllegalArgumentException if a capacity, term type or requested isolation level is unsupported + * @throws NullPointerException if an explicitly required argument or mapped value is null + */ + private static Value immutable(final Value value) { + Objects.requireNonNull(value, "value"); + if (value instanceof Literal literal && !(literal.getDatatype() instanceof SimpleIRI)) { + final IRI datatype = (IRI) immutable(literal.getDatatype()); + if (value instanceof SimpleLiteral) { + return literal.getLanguage().map(language -> (Value) new SimpleLiteral(literal.getLabel(), language)) + .orElseGet(() -> new SimpleLiteral(literal.getLabel(), datatype)); + } + return Values.factory().createLiteral(literal.getLabel(), datatype); + } + if (value instanceof SimpleIRI || value instanceof SimpleBNode || value instanceof SimpleLiteral + || value instanceof SimpleInteger || value instanceof SimpleDecimal || value instanceof SimpleDouble + || value instanceof SimpleDate || value instanceof SimpleDateTime || value instanceof SimpleTime + || value instanceof SimpleDuration) { + return value; + } + if (value instanceof IRI iri) { + return new SimpleIRI(iri.stringValue()); + } + if (value instanceof BNode blank) { + return new SimpleBNode(blank.getID()); + } + if (value instanceof Literal literal) { + return literal.getLanguage().map(language -> (Value) new SimpleLiteral(literal.getLabel(), language)) + .orElseGet(() -> new SimpleLiteral(literal.getLabel(), (IRI) immutable(literal.getDatatype()))); + } + if (value instanceof Triple triple) { + return new SimpleTriple((Resource) immutable(triple.subject()), (IRI) immutable(triple.predicate()), + immutable(triple.object())); + } + throw new IllegalArgumentException("Unsupported RDF term: " + value.getClass().getName()); + } + + /** + * Captures RDF identities without changing the statement or pattern semantics. + * + * @param statement statement whose RDF identities have been captured + * @return the original supported native value or a captured representation of its RDF identity + * @throws NullPointerException if an explicitly required argument or mapped value is null + */ + private static Statement immutable(final Statement statement) { + Objects.requireNonNull(statement, "statement"); + final Resource context = statement.getContext(); + final Resource subject = (Resource) immutable(statement.getSubject()); + final IRI predicate = (IRI) immutable(statement.getPredicate()); + final Value object = immutable(statement.getObject()); + final Resource graph = context == null ? null : (Resource) immutable(context); + if (statement.getClass() == SimpleStatement.class && subject == statement.getSubject() + && predicate == statement.getPredicate() && object == statement.getObject() && graph == context) { + return statement; + } + return new SimpleStatement(subject, predicate, object, graph); + } + + /** + * Share safe native terms without changing lexical identity or value implementation. + * + * @param statement statement whose RDF identities have been captured + * @param subject canonical subject key, or null for a wildcard search + * @param predicate canonical predicate key; null denotes a wildcard for search operations + * @param object canonical object key, or null for a wildcard search + * @return the original statement when all values are already shared, otherwise a statement using compatible + * canonical values + */ + static Statement sharedStatement(final Statement statement, final TermKey subject, final TermKey predicate, + final TermKey object) { + return sharedStatement(statement, subject, predicate, object, statement.getContext()); + } + + /** + * Reuses compatible canonical RDF values without changing lexical identity or implementation type. + * + * @param statement statement whose RDF identities have been captured + * @param subject canonical subject key, or null for a wildcard search + * @param predicate canonical predicate key; null denotes a wildcard for search operations + * @param object canonical object key, or null for a wildcard search + * @param context canonical graph identifier; null denotes the default graph + * @return the original statement when all values are already shared, otherwise a statement using compatible + * canonical values + */ + static Statement sharedStatement(final Statement statement, final TermKey subject, final TermKey predicate, + final TermKey object, final Resource context) { + final Resource s = (Resource) sharedValue(statement.getSubject(), subject.value()); + final IRI p = (IRI) sharedValue(statement.getPredicate(), predicate.value()); + final Value o = sharedValue(statement.getObject(), object.value()); + final Resource g = context == null ? null : (Resource) sharedValue(statement.getContext(), context); + return s == statement.getSubject() && p == statement.getPredicate() && o == statement.getObject() + && g == statement.getContext() ? statement : new SimpleStatement(s, p, o, g); + } + + /** + * Reuses a compatible immutable scalar value while preserving its implementation and datatype. + * + * @param original original captured RDF value + * @param canonical compatible canonical RDF value + * @return the compatible canonical value, or the original value when pooling is unsuitable + */ + private static Value sharedValue(final Value original, final Value canonical) { + if (original == canonical || original.getClass() != canonical.getClass()) { + return original; + } + if (original instanceof Literal literal && canonical instanceof Literal other + && !sameLiteralImplementation(literal, other)) { + return original; + } + if (original instanceof SimpleIRI iri && canonical instanceof SimpleIRI other) { + return iri.getNamespace().equals(other.getNamespace()) && iri.getLocalName().equals(other.getLocalName()) + ? canonical + : original; + } + if (original instanceof SimpleLiteral literal && canonical instanceof SimpleLiteral other) { + return literal.getCoreDatatype() == other.getCoreDatatype() ? canonical : original; + } + // Calendar-backed terms and nested triples are deliberately not pooled. + return original instanceof SimpleBNode || original instanceof SimpleInteger || original instanceof SimpleDecimal + || original instanceof SimpleDouble || original instanceof SimpleDuration ? canonical : original; + } + + /** + * Checks literal datatype implementation compatibility using the existing comparison order. + * + * @param literal original captured literal + * @param other value or search object to compare with the stored key + * @return true when datatype implementation and native components permit sharing + */ + private static boolean sameLiteralImplementation(final Literal literal, final Literal other) { + if (literal.getCoreDatatype() != other.getCoreDatatype()) { + return false; + } + final IRI datatype = literal.getDatatype(); + final IRI canonicalDatatype = other.getDatatype(); + + return datatype == canonicalDatatype || (datatype.getClass() == canonicalDatatype.getClass() + && datatype.getNamespace().equals(canonicalDatatype.getNamespace()) + && datatype.getLocalName().equals(canonicalDatatype.getLocalName())); + } + + /** + * Captures RDF identities without changing the statement or pattern semantics. + * + * @param pattern captured search pattern + * @return the original supported native value or a captured representation of its RDF identity + * @throws NullPointerException if an explicitly required argument or mapped value is null + */ + private static StatementPattern immutable(final StatementPattern pattern) { + Objects.requireNonNull(pattern, "pattern"); + final Resource[] contexts = pattern.getContexts(); + for (int i = 0; i < contexts.length; i++) { + if (contexts[i] != null) { + contexts[i] = (Resource) immutable(contexts[i]); + } + } + return StatementPattern.of(pattern.getSubject() == null ? null : (Resource) immutable(pattern.getSubject()), + pattern.getPredicate() == null ? null : (IRI) immutable(pattern.getPredicate()), + pattern.getObject() == null ? null : immutable(pattern.getObject()), contexts); + } + + /** + * Captures the selected partitions immediately and traverses matches lazily. + * + *

+ * Runtime failures and Errors during cursor construction release the captured ownership before the original failure + * is rethrown. Error handling is limited to resource release; no failure is recovered or converted. + * + *

+ * The returned sequential stream must be used in try-with-resources. Terminal operations do not close streams: + * early operations such as {@code findFirst()} or {@code limit()} can retain ownership, and a sized {@code count()} + * may skip traversal entirely. Exhaustion and traversal failures release ownership as an additional safeguard; + * explicit close remains the caller's responsibility. + * + * @param pattern captured search pattern + * @return a sequential stream owning its read snapshot; the caller must close it + * @throws IllegalStateException if the session is closed or its transaction belongs to another thread + */ + @SuppressWarnings("java:S1181") + @Override + public Stream find(StatementPattern pattern) { + pattern = immutable(pattern); + synchronized (database) { + checkAccess(); + final State snapshot = state().copy(pattern.getContexts()); + try { + var cursor = new SnapshotCursor(snapshot, pattern); + return StreamSupport.stream(cursor, false).onClose(cursor::close); + } catch (final RuntimeException | Error failure) { + snapshot.release(); + throw failure; + } + } + } + + /** + * Single-consumer cursor owning a captured state until close, exhaustion or failure. Known-size traversal can + * release ownership before delivering the last immutable statement. Close is idempotent and clears traversal + * references under the monitor. No database monitor is held while visiting rows or invoking consumer code. Session + * and transaction closure do not invalidate this independent read snapshot. + */ + private final class SnapshotCursor implements StatementSpliterator, AutoCloseable { + /** + * Read state retained until this cursor is closed or exhausted. + */ + private State snapshot; + /** + * Ordered traversal of the retained partition registry. + */ + private Iterator partitions; + /** + * Positional traversal of the current retained partition. + */ + private TripleIndex.Cursor rows; + /** + * Captured immutable search pattern. + */ + private final StatementPattern pattern; + /** + * Canonical traversal subject or current batch subject, according to the enclosing traversal. + */ + private final TermKey subject; + /** + * Canonical traversal predicate or current batch predicate, according to the enclosing traversal. + */ + private final TermKey predicate; + /** + * Captured object search key; null denotes a wildcard. + */ + private final TermKey object; + /** + * Whether the retained match count is known exactly. + */ + private final boolean sized; + /** + * Snapshot-bound guarantee of SPO uniqueness independently of context. + */ + private final boolean distinctTriples; + /** + * Remaining exact count or the unsized upper bound while this cursor is open. + */ + private long remaining; + + /** + * Creates a single-consumer cursor owning the supplied retained read state. + * + * @param snapshot retained read state + * @param pattern captured search pattern + */ + private SnapshotCursor(final State snapshot, final StatementPattern pattern) { + this.snapshot = snapshot; + this.pattern = pattern; + distinctTriples = hasAtMostOnePopulatedPartition(snapshot); + subject = TermKey.nullable(pattern.getSubject()); + predicate = TermKey.nullable(pattern.getPredicate()); + object = TermKey.nullable(pattern.getObject()); + sized = subject == null && predicate == null && object == null; + remaining = sized ? snapshot.size : Long.MAX_VALUE; + partitions = snapshot.graphs.values().iterator(); + } + + /** + * {@inheritDoc} + * + * @param action consumer invoked outside the database monitor + * @return true when a statement was delivered, false when this cursor is closed or exhausted + * @throws NullPointerException if an explicitly required argument or mapped value is null + */ + // Traversal Errors release ownership and are rethrown unchanged. + @SuppressWarnings("java:S1181") + @Override + public boolean tryAdvance(final Consumer action) { + Objects.requireNonNull(action); + if (snapshot == null) { + return false; + } + try { + for (;;) { + final Statement statement = rows == null ? null : rows.next(); + if (statement != null) { + if (sized && --remaining == 0) { + close(); + } + action.accept(statement); + return true; + } + if (!partitions.hasNext()) { + close(); + return false; + } + rows = partitions.next().index(pattern).cursor(subject, predicate, object); + } + } catch (final RuntimeException | Error failure) { + close(); + throw failure; + } + } + + /** + * {@inheritDoc} + * + * @param action consumer invoked outside the database monitor + * @throws NullPointerException if an explicitly required argument or mapped value is null + */ + @Override + public void forEachRemaining(final Consumer action) { + Objects.requireNonNull(action); + if (snapshot == null) { + return; + } + // A complete scan uses the existing bulk visitor, without per-row cursor overhead. + if (rows != null) { + StatementSpliterator.super.forEachRemaining(action); + return; + } + try { + final Predicate visitor = statement -> { + if (sized && --remaining == 0) { + close(); + } + action.accept(statement); + return snapshot == null; + }; + while (snapshot != null && partitions.hasNext()) { + partitions.next().index(pattern).visit(subject, predicate, object, visitor); + } + } finally { + close(); + } + } + + /** + * {@inheritDoc} + * + * @return true only when SPO uniqueness is guaranteed for this retained read + */ + @Override + public boolean hasDistinctTriples() { + return distinctTriples; + } + + /** + * {@inheritDoc} + * + * @return null; this traversal cannot be split + */ + @Override + public Spliterator trySplit() { + return null; + } + + /** + * {@inheritDoc} + * + * @return remaining size estimate, or zero once the cursor is closed + */ + @Override + public long estimateSize() { + return snapshot == null ? 0 : remaining; + } + + /** + * {@inheritDoc} + * + * @return the ORDERED, NONNULL and IMMUTABLE flags, plus SIZED and SUBSIZED when the exact count is known + */ + @Override + public int characteristics() { + return ORDERED | NONNULL | IMMUTABLE | (sized ? SIZED | SUBSIZED : 0); + } + + /** + * {@inheritDoc} + */ + @Override + public void close() { + synchronized (database) { + if (snapshot == null) { + return; + } + snapshot.release(); + snapshot = null; + partitions = null; + rows = null; + } + } + } + + /** + * Empty named partitions survive CLEAR but cannot contribute duplicate triples. + * + * @param snapshot retained read state + * @return true when at most one selected partition contains statements + */ + private static boolean hasAtMostOnePopulatedPartition(final State snapshot) { + if (snapshot.graphs.size() <= 1) { + return true; + } + boolean populated = false; + for (final Partition partition : snapshot.graphs.values()) { + if (partition.spo.size() == 0) { + continue; + } + if (populated) { + return false; + } + populated = true; + } + return true; + } + + /** + * {@inheritDoc} + * + * @param pattern captured search pattern + * @return matching statement count + * @throws IllegalStateException if the session is closed or its transaction belongs to another thread + */ + @Override + public long count(StatementPattern pattern) { + pattern = immutable(pattern); + synchronized (database) { + checkAccess(); + if (pattern.matchesAll()) { + return state().size; + } + final State current = state(); + if (current.graphs.isEmpty()) { + return 0; + } + final Resource[] contexts = pattern.getContexts(); + // Keep one-partition keys local to the index for JVM scalar replacement. + if (contexts.length == 1) { + final Partition partition = current.graphs.get(contexts[0]); + return partition == null ? 0 : partition.index(pattern).count(pattern); + } + if (contexts.length == 0 && current.graphs.size() == 1) { + final Partition partition = current.graphs.values().iterator().next(); + return partition.index(pattern).count(pattern); + } + return countSelectedPartitions(current, contexts, pattern); + } + } + + /** + * Counts selected graph matches using one set of call-local search keys. + * + * @param current session-visible state guarded by the database monitor + * @param contexts selected graph identifiers; an empty array selects all graphs + * @param pattern captured search pattern + * @return number of matching statements across selected graph partitions + */ + private static long countSelectedPartitions(final State current, final Resource[] contexts, + final StatementPattern pattern) { + // Call-local keys: no per-partition hashing or retained query cache. + final TermKey subject = TermKey.nullable(pattern.getSubject()); + final TermKey predicate = TermKey.nullable(pattern.getPredicate()); + final TermKey object = TermKey.nullable(pattern.getObject()); + long count = 0; + if (contexts.length == 0) { + for (final Partition partition : current.graphs.values()) { + count += partition.index(pattern).count(subject, predicate, object); + } + } else { + for (final Resource context : contexts) { + final Partition partition = current.graphs.get(context); + if (partition != null) { + count += partition.index(pattern).count(subject, predicate, object); + } + } + } + return count; + } + + /** + * {@inheritDoc} + * + * @param pattern captured search pattern + * @return true when at least one matching statement exists + * @throws IllegalStateException if the session is closed or its transaction belongs to another thread + */ + @Override + public boolean contains(StatementPattern pattern) { + pattern = immutable(pattern); + synchronized (database) { + checkAccess(); + return visit(state(), pattern, s -> true); + } + } + + /** + * Traverses matching statements or live values until the visitor requests an early stop. + * + * @param state state whose ownership is guarded by the database monitor + * @param pattern captured search pattern + * @param visitor non-mutating callback; true requests an early stop for predicate visitors + * @return true when a matching value caused the visitor to request an early stop + */ + private static boolean visit(final State state, final StatementPattern pattern, + final Predicate visitor) { + if (state.graphs.isEmpty()) { + return false; + } + final Resource[] contexts = pattern.getContexts(); + if (contexts.length == 1) { + final Partition partition = state.graphs.get(contexts[0]); + return partition != null && partition.index(pattern).visit(pattern, visitor); + } + if (contexts.length == 0 && state.graphs.size() == 1) { + final Partition partition = state.graphs.values().iterator().next(); + return partition.index(pattern).visit(pattern, visitor); + } + return visitSelectedPartitions(state, pattern, visitor, contexts); + } + + /** + * Visits selected graphs using call-local keys and stable registry order. + * + * @param state state whose ownership is guarded by the database monitor + * @param pattern captured search pattern + * @param visitor non-mutating callback; true requests an early stop for predicate visitors + * @param contexts selected graph identifiers; an empty array selects all graphs + * @return true when the visitor requests an early stop on a matching statement + */ + private static boolean visitSelectedPartitions(final State state, final StatementPattern pattern, + final Predicate visitor, final Resource[] contexts) { + final TermKey subject = TermKey.nullable(pattern.getSubject()); + final TermKey predicate = TermKey.nullable(pattern.getPredicate()); + final TermKey object = TermKey.nullable(pattern.getObject()); + if (contexts.length == 0) { + for (final Partition partition : state.graphs.values()) { + if (partition.index(pattern).visit(subject, predicate, object, visitor)) { + return true; + } + } + } else { + final List partitions = selectPartitions(state, contexts); + for (final Partition partition : partitions) { + if (partition.index(pattern).visit(subject, predicate, object, visitor)) { + return true; + } + } + } + return false; + } + + /** + * Materializes selected partitions in registry order; the database monitor must be held. + * + * @param state state whose ownership is guarded by the database monitor + * @param contexts selected graph identifiers; an empty array selects all graphs + * @return existing selected partitions in stable registry order + */ + private static List selectPartitions(final State state, final Resource[] contexts) { + // Preserve registry order independently of the caller's context ordering. + final List partitions = new ArrayList<>(); + for (final Resource context : contexts) { + final Partition partition = state.graphs.get(context); + if (partition != null) { + partitions.add(partition); + } + } + partitions.sort(Comparator.comparingLong(p -> p.ordinal)); + return partitions; + } + + /** + * {@inheritDoc} + * + * @param statement statement whose RDF identities have been captured + * @return true when an absent entry was added, false when it was already present + * @throws IllegalStateException if the session is closed or its transaction belongs to another thread, or a + * conflicting batch is active + */ + @Override + public boolean add(final Statement statement) { + final Statement captured = immutable(statement); + synchronized (database) { + checkWrite(); + final State current = state(); + final Partition existing = current.graphs.get(captured.getContext()); + if (existing != null && existing.references > 1 && existing.contains(captured)) { + return false; + } + Partition partition = current.writable(captured.getContext()); + if (partition == null) { + partition = new Partition(captured.getContext(), current.nextOrdinal++); + current.graphs.put(captured.getContext(), partition); + } + final boolean added = partition.add(captured); + if (added) { + state().size++; + } + changed(added); + return added; + } + } + + /** + * {@inheritDoc} + * + * @param statement statement whose RDF identities have been captured + * @return true when an entry was removed, false when it was absent + * @throws IllegalStateException if the session is closed or its transaction belongs to another thread, or a + * conflicting batch is active + */ + @Override + public boolean remove(final Statement statement) { + final Statement captured = immutable(statement); + synchronized (database) { + checkWrite(); + final Partition existing = state().graphs.get(captured.getContext()); + if (existing == null || (existing.references > 1 && !existing.contains(captured))) { + return false; + } + final Partition partition = state().writable(captured.getContext()); + final boolean removed = partition != null && partition.remove(captured); + if (removed) { + state().size--; + } + changed(removed); + return removed; + } + } + + /** + * {@inheritDoc} + * + * @param pattern captured search pattern + * @return number of matching statements removed + * @throws IllegalStateException if the session is closed or its transaction belongs to another thread, or a + * conflicting batch is active + */ + @Override + public long remove(StatementPattern pattern) { + pattern = immutable(pattern); + synchronized (database) { + checkWrite(); + final State current = state(); + final Resource[] contexts = pattern.getContexts(); + long removed = 0; + if (contexts.length == 0) { + // Replacing an existing value does not change LinkedHashMap iteration order. + for (final Partition partition : current.graphs.values()) { + removed += remove(current, partition, pattern); + } + } else { + for (final Resource context : contexts) { + final Partition partition = current.graphs.get(context); + if (partition != null) { + removed += remove(current, partition, pattern); + } + } + } + current.size -= removed; + changed(removed != 0); + return removed; + } + } + + /** + * Removes matching content while preserving the order of survivors. + * + * @param current session-visible state guarded by the database monitor + * @param partition partition retained or exclusively owned by the supplied state + * @param pattern captured search pattern + * @return number of matching statements removed + * @throws IllegalStateException if the session is closed or its transaction belongs to another thread, or a + * conflicting batch is active + */ + private static long remove(final State current, final Partition partition, final StatementPattern pattern) { + final long size = partition.spo.size(); + if (size == 0) { + return 0; + } + final long count = pattern.isSubjectWildcard() && pattern.isPredicateWildcard() && pattern.isObjectWildcard() + ? size + : partition.index(pattern).count(pattern); + if (count == 0) { + return 0; + } + if (count == size) { + current.empty(partition); + } else if (count >= 64 && count >= size / 4) { + // Broad deletions filter each index once; selective ones keep indexed seeks. + current.writable(partition.context).removeMatches(pattern); + } else { + final List matches = new ArrayList<>(); + partition.index(pattern).visit(pattern, s -> { + matches.add(s); + return false; + }); + final Partition writable = current.writable(partition.context); + for (final Statement statement : matches) { + writable.remove(statement); + } + } + return count; + } + + /** + * {@inheritDoc} + * + * @param contexts selected graph identifiers; an empty array selects all graphs + * @return number of removed statements + * @throws IllegalStateException if the session is closed or its transaction belongs to another thread, or a + * conflicting batch is active + */ + @Override + public long clear(final Resource... contexts) { + return remove(StatementPattern.of(null, null, null, contexts)); + } + + /** + * {@inheritDoc} + * + * @param graph named graph identifier or staging registry key, according to the operation + * @return true when the named graph was created + * @throws NullPointerException if an explicitly required argument or mapped value is null + * @throws IllegalStateException if the session is closed or its transaction belongs to another thread, or a + * conflicting batch is active + */ + @Override + public boolean createGraph(final Resource graph) { + final Resource captured = (Resource) immutable(Objects.requireNonNull(graph, "graph")); + synchronized (database) { + checkWrite(); + if (state().graphs.containsKey(captured)) { + return false; + } + state().graphs.put(captured, new Partition(captured, state().nextOrdinal++)); + changed(true); + return true; + } + } + + /** + * {@inheritDoc} + * + * @param graph named graph identifier or staging registry key, according to the operation + * @return true when the named graph existed and was dropped + * @throws NullPointerException if an explicitly required argument or mapped value is null + * @throws IllegalStateException if the session is closed or its transaction belongs to another thread, or a + * conflicting batch is active + */ + @Override + public boolean dropGraph(final Resource graph) { + final Resource captured = (Resource) immutable(Objects.requireNonNull(graph, "graph")); + synchronized (database) { + checkWrite(); + final Partition removed = state().graphs.remove(captured); + if (removed == null) { + return false; + } + removed.references--; + state().size -= removed.spo.size(); + changed(true); + return true; + } + } + + /** + * Atomic finite batch, including graph creation; source is consumed once outside the database lock. + * + * @param statements finite source consumed in exactly one pass + * @return number of newly added statements across the atomic batch + * @throws StorageException if publication fails or a concurrent committed version prevents commit + * @throws NullPointerException if an explicitly required argument or mapped value is null + * @throws IllegalStateException if the session is closed or its transaction belongs to another thread, or a + * conflicting batch is active + */ + @Override + public long addAll(final Iterable statements) { + Objects.requireNonNull(statements, "statements"); + synchronized (database) { + checkWrite(); + loading = true; + } + try { + return publishBatch(stageStatements(statements)); + } catch (final NullPointerException failure) { + throw failure; + } catch (final RuntimeException failure) { + throw new StorageException(ErrorCode.MUTATION_FAILED, "Atomic batch failed", failure); + } finally { + synchronized (database) { + loading = false; + } + } + } + + /** + * Captures one pass over the source in private SPO staging; no database monitor is held. + * + * @param statements finite source consumed in exactly one pass + * @return private graph-local SPO staging in source encounter order + */ + private static Map stageStatements(final Iterable statements) { + final int expectedTerms = stagingTermHint(statements); + final Map staged = new LinkedHashMap<>(); + final Function newPartition = graph -> new StagedPartition(graph, + TripleIndex.forStaging(staged.isEmpty() ? expectedTerms : 0)); + for (final Statement statement : statements) { + final Statement captured = immutable(statement); + final StagedPartition partition = staged.computeIfAbsent(captured.getContext(), newPartition); + partition.statements.add(captured, partition.context); + } + return staged; + } + + /** + * Merges completed staging and atomically publishes it under the database monitor. + * + * @param staged unpublished graph-local SPO staging + * @return number of newly added statements + */ + private long publishBatch(final Map staged) { + synchronized (database) { + checkAccess(); + final State target = state(); + final State replacement = target.copy(); + boolean published = false; + try { + final long[] added = { 0 }; + staged.forEach((graph, stagedPartition) -> { + added[0] += mergeStagedPartition(target, replacement, graph, stagedPartition); + }); + if (added[0] != 0) { + replacement.size += added[0]; + target.release(); + if (transaction == null) { + database.committed = replacement; + } else { + transaction.state = replacement; + } + published = true; + changed(true); + } + return added[0]; + } finally { + if (!published) { + replacement.release(); + } + } + } + } + + /** + * Builds one changed partition in the unpublished replacement state. + * + * @param target current state being merged without mutating its retained partitions + * @param replacement unpublished replacement state owning the changed partitions + * @param graph named graph identifier or staging registry key, according to the operation + * @param stagedPartition private SPO staging for this graph + * @return number of new statements in this graph + */ + private static long mergeStagedPartition(final State target, final State replacement, final Resource graph, + final StagedPartition stagedPartition) { + final TripleIndex additions = stagedPartition.statements; + final Partition existing = target.graphs.get(graph); + final TripleIndex merged = existing == null ? additions : existing.spo.copy(); + final long before = merged.size(); + if (existing != null) { + additions.visit(StatementPattern.matchAll(), s -> { + merged.add(s, existing.context); + return false; + }); + } + final long count = existing == null ? merged.size() : merged.size() - before; + if (count != 0) { + final Partition built = Partition.build(merged, existing == null ? graph : existing.context, + existing == null ? replacement.nextOrdinal++ : existing.ordinal); + if (existing != null) { + existing.references--; + } + replacement.graphs.put(graph, built); + } + return count; + } + + /** + * Computes a bounded reservation from an available Collection size without iterating the source. + * + * @param statements finite source consumed in exactly one pass + * @return bounded term reservation, or zero when the size is unavailable + */ + private static int stagingTermHint(final Iterable statements) { + if (statements instanceof Collection collection) { + try { + return (int) Math.min(MAX_INITIAL_STAGING_TERMS, 3L * collection.size()); + } catch (final UnsupportedOperationException ignored) { + /* Some streaming collections cannot report a size. */ } + } + return 0; + } + + /** + * Batch-local context and SPO staging; never published or shared with reader states. + * + * @param context canonical graph identifier; null denotes the default graph + * @param statements unpublished SPO staging + */ + private record StagedPartition(/** + * Canonical graph identifier; null denotes the default graph. + */ + Resource context, /** + * Unpublished graph-local SPO staging. + */ + TripleIndex statements) { + } + + /** + * {@inheritDoc} + * + * @return an immutable ordered set of named graph identifiers + * @throws IllegalStateException if the session is closed or its transaction belongs to another thread + */ + @Override + public Set getContexts() { + synchronized (database) { + checkAccess(); + final Set contexts = new LinkedHashSet<>(state().graphs.keySet()); + contexts.remove(null); + return Collections.unmodifiableSet(contexts); + } + } + + /** + * {@inheritDoc} + * + * @return an immutable ordered set of distinct subjects + * @throws IllegalStateException if the session is closed or its transaction belongs to another thread + */ + @Override + public Set getSubjects() { + synchronized (database) { + checkAccess(); + final Set subjects = new LinkedHashSet<>(); + state().graphs.values().forEach(p -> p.spo.firstKeys().forEach(v -> subjects.add((Resource) v))); + return Collections.unmodifiableSet(subjects); + } + } + + /** + * {@inheritDoc} + * + * @return an immutable ordered set of distinct predicates + * @throws IllegalStateException if the session is closed or its transaction belongs to another thread + */ + @Override + public Set getPredicates() { + synchronized (database) { + checkAccess(); + final Set predicates = new LinkedHashSet<>(); + state().graphs.values().forEach(p -> p.pos.firstKeys().forEach(v -> predicates.add((IRI) v))); + return Collections.unmodifiableSet(predicates); + } + } + + /** + * {@inheritDoc} + * + * @return an immutable ordered set of distinct objects + * @throws IllegalStateException if the session is closed or its transaction belongs to another thread + */ + @Override + public Set getObjects() { + synchronized (database) { + checkAccess(); + final Set objects = new LinkedHashSet<>(); + state().graphs.values().forEach(p -> objects.addAll(p.osp.firstKeys())); + return Collections.unmodifiableSet(objects); + } + } + + /** + * {@inheritDoc} + * + * @return exact cardinalities of the session-visible state + * @throws IllegalStateException if the session is closed or its transaction belongs to another thread + */ + @Override + public StorageStatistics getStatistics() { + synchronized (database) { + checkAccess(); + return new StorageStatistics(state().size, getSubjects().size(), getPredicates().size(), + getObjects().size(), getContexts().size()); + } + } + + /** + * {@inheritDoc} + * + * @return true + */ + @Override + public boolean supportsTransactions() { + return true; + } + + /** + * {@inheritDoc} + * + * @return the immutable set containing SERIALIZABLE + */ + @Override + public Set getSupportedIsolationLevels() { + return Set.of(IsolationLevel.SERIALIZABLE); + } + + /** + * {@inheritDoc} + * + * @return the active transaction owned by the calling thread + * @throws IllegalStateException if the session is closed or its transaction belongs to another thread, or a + * conflicting batch is active + */ + @Override + public Transaction beginTransaction() { + return beginTransaction(IsolationLevel.SERIALIZABLE); + } + + /** + * {@inheritDoc} + * + * @param level requested supported isolation level + * @return the active transaction owned by the calling thread + * @throws IllegalArgumentException if a capacity, term type or requested isolation level is unsupported + * @throws IllegalStateException if the required representation, ownership or lifecycle state is not satisfied + */ + @Override + public Transaction beginTransaction(final IsolationLevel level) { + if (!getSupportedIsolationLevels().contains(level)) { + throw new IllegalArgumentException("Unsupported isolation level: " + level); + } + synchronized (database) { + checkWrite(); + if (transaction != null) { + throw new IllegalStateException("A transaction is already active on this session"); + } + transaction = new SnapshotTransaction(database.committed.copy(), database.version); + return transaction; + } + } + + /** + * Thread-owned optimistic transaction retaining a captured state until commit or rollback. + */ + private final class SnapshotTransaction implements Transaction { + /** + * Stable transaction identifier. + */ + private final String id = UUID.randomUUID().toString(); + /** + * Owning manager flag or transaction thread, according to the enclosing class. + */ + private final Thread owner = Thread.currentThread(); + /** + * Committed version captured for optimistic conflict detection. + */ + private final long version; + /** + * Transaction-owned state until publication or release. + */ + private State state; + /** + * Whether this transaction changed its captured state. + */ + private boolean dirty; + /** + * Current transaction lifecycle status. + */ + private TransactionState status = TransactionState.ACTIVE; + + /** + * Creates a thread-owned transaction for the captured committed version. + * + * @param state state whose ownership is guarded by the database monitor + * @param version committed version captured for conflict detection + */ + private SnapshotTransaction(final State state, final long version) { + this.state = state; + this.version = version; + } + + /** + * Requires access from the thread that began the transaction. + * + * @throws IllegalStateException if the required representation, ownership or lifecycle state is not satisfied + */ + private void checkOwner() { + if (Thread.currentThread() != owner) { + throw new IllegalStateException("Transaction belongs to another thread"); + } + } + + /** + * Requires the transaction owner, active status and no overlapping session batch. + * + * @throws IllegalStateException if the required representation, ownership or lifecycle state is not satisfied + */ + private void requireActive() { + checkOwner(); + if (status != TransactionState.ACTIVE) { + throw new IllegalStateException("Transaction is inactive"); + } + if (loading) { + throw new IllegalStateException("A batch is active on this session"); + } + } + + /** + * {@inheritDoc} + * + * @return the transaction identifier + */ + @Override + public String getId() { + return id; + } + + /** + * {@inheritDoc} + * + * @return true while the transaction is active + */ + @Override + public boolean isActive() { + synchronized (database) { + return status == TransactionState.ACTIVE; + } + } + + /** + * {@inheritDoc} + * + * @return the current lifecycle or transaction status + */ + @Override + public TransactionState getState() { + synchronized (database) { + return status; + } + } + + /** + * {@inheritDoc} + * + * @throws StorageException if publication fails or a concurrent committed version prevents commit + */ + @Override + public void commit() { + synchronized (database) { + requireActive(); + checkAccess(); + if (database.version != version) { + throw new StorageException(ErrorCode.TRANSACTION_ERROR, + "Concurrent storage modification; roll back and retry"); + } + if (dirty) { + database.committed.release(); + database.committed = state; + state = null; // Ownership transferred to the committed state. + database.version++; + } + finish(TransactionState.COMMITTED); + } + } + + /** + * {@inheritDoc} + */ + @Override + public void rollback() { + synchronized (database) { + requireActive(); + finish(TransactionState.ROLLED_BACK); + } + } + + /** + * Sets terminal transaction status and releases any state ownership not transferred at commit. + * + * @param terminal terminal transaction status + */ + private void finish(final TransactionState terminal) { + status = terminal; + if (state != null) { + state.release(); + } + state = null; + transaction = null; + } + + /** + * {@inheritDoc} + */ + @Override + public void close() { + synchronized (database) { + if (isActive()) { + rollback(); + } + } + } + } + +} diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/MemoryLifecycleManager.java b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/MemoryLifecycleManager.java deleted file mode 100644 index 6c3e95f9d..000000000 --- a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/MemoryLifecycleManager.java +++ /dev/null @@ -1,91 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.memory; - -import fr.inria.corese.core.next.storage.api.lifecycle.LifecycleState; -import fr.inria.corese.core.next.storage.api.lifecycle.StorageLifecycle; -import fr.inria.corese.core.next.storage.api.config.StorageConfig; -import fr.inria.corese.core.next.storage.api.exception.ErrorCode; -import fr.inria.corese.core.next.storage.api.exception.StorageException; - -import java.util.Optional; -import java.util.concurrent.atomic.AtomicReference; - -/** - * Lifecycle manager for {@link MemoryStorageManager}. - */ -final class MemoryLifecycleManager implements StorageLifecycle { - - private volatile LifecycleState state = LifecycleState.NOT_INITIALIZED; - private final AtomicReference config = new AtomicReference<>(); - - /** - * Constructs a new MemoryLifecycleManager. - * - * @param adapter the InMemoryStatementStore to manage (must not be null) - * @throws IllegalArgumentException if adapter is null - */ - public MemoryLifecycleManager(InMemoryStatementStore adapter) { - if (adapter == null) throw new IllegalArgumentException("InMemoryStatementStore cannot be null"); - } - - /** - * Initializes the MemoryStorageManager with the given configuration. - * - * @param config the storage configuration (must not be null) - * @throws IllegalArgumentException if config is null - * @throws IllegalStateException if already initialized - * @throws StorageException if initialization fails - */ - @Override - public synchronized void initialize(StorageConfig config) throws StorageException { - if (config == null) throw new IllegalArgumentException("Config cannot be null"); - if (state != LifecycleState.NOT_INITIALIZED) { - throw new IllegalStateException("Already initialized"); - } - - this.state = LifecycleState.INITIALIZING; - try { - this.config.set(config); - this.state = LifecycleState.RUNNING; - } catch (Exception e) { - this.state = LifecycleState.NOT_INITIALIZED; - throw new StorageException(ErrorCode.INITIALIZATION_FAILED, "Init failed", e); - } - } - - /** - * Shuts down the MemoryStorageManager. - * - * @throws IllegalStateException if not currently running - * @throws StorageException if shutdown fails - */ - @Override - public synchronized void shutdown() throws StorageException { - if (state != LifecycleState.RUNNING) { - throw new IllegalStateException("Not running"); - } - // Note: We don't clear data on shutdown - caller must do that explicitly if desired - this.state = LifecycleState.SHUTTING_DOWN; - this.state = LifecycleState.SHUTDOWN; - this.config.set(null); - } - - /** - * Returns the current lifecycle state. - * - * @return the current state (never null) - */ - @Override - public LifecycleState getState() { - return state; - } - - /** - * Returns the current configuration. - * - * @return the config if initialized/running, empty otherwise - */ - @Override - public Optional getConfig() { - return Optional.ofNullable(config.get()); - } -} diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/MemoryMetadataOperations.java b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/MemoryMetadataOperations.java deleted file mode 100644 index c8c91a45e..000000000 --- a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/MemoryMetadataOperations.java +++ /dev/null @@ -1,88 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.memory; - -import fr.inria.corese.core.next.data.api.term.IRI; -import fr.inria.corese.core.next.data.api.term.Resource; -import fr.inria.corese.core.next.data.api.term.Value; -import fr.inria.corese.core.next.storage.api.operations.MetadataOperations; -import fr.inria.corese.core.next.storage.api.exception.StorageException; -import fr.inria.corese.core.next.storage.api.model.StorageStatistics; - -import java.util.Collections; -import java.util.Set; - -/** - * Metadata operations implementation for {@link MemoryStorageManager}. - */ -final class MemoryMetadataOperations implements MetadataOperations { - - private final InMemoryStatementStore adapter; - - /** - * Constructs a new MemoryMetadataOperations. - * - * @param adapter the InMemoryStatementStore for data access (must not be null) - * @throws IllegalArgumentException if adapter is null - */ - public MemoryMetadataOperations(InMemoryStatementStore adapter) { - if (adapter == null) throw new IllegalArgumentException("InMemoryStatementStore cannot be null"); - this.adapter = adapter; - } - - /** - * Returns all unique predicate IRIs in the store. - * - * @return unmodifiable set of all predicates (never null) - * @throws StorageException if the operation fails - */ - @Override - public Set getPredicates() throws StorageException { - return Collections.unmodifiableSet(adapter.getPredicates()); - } - - /** - * Returns all unique subject resources in the store. - * - * @return unmodifiable set of all subjects (never null) - */ - @Override - public Set getSubjects() { - return Collections.unmodifiableSet(adapter.getSubjects()); - } - - /** - * Returns all unique object values in the store. - * - * @return unmodifiable set of all objects (never null) - */ - @Override - public Set getObjects() { - return Collections.unmodifiableSet(adapter.getObjects()); - } - - /** - * Returns all unique context (named graph) identifiers in the store. - * - * @return unmodifiable set of all contexts (never null) - * @throws StorageException if the operation fails - */ - @Override - public Set getContexts() throws StorageException { - return Collections.unmodifiableSet(adapter.getContexts()); - } - - /** - * Returns aggregate statistics about the store. - * - * @return a {@link StorageStatistics} record with aggregate counts - */ - @Override - public StorageStatistics getStatistics() { - return new StorageStatistics( - adapter.size(), - adapter.getSubjects().size(), - adapter.getPredicates().size(), - adapter.getObjects().size(), - adapter.getContexts().size() - ); - } -} diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/MemoryMutationOperations.java b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/MemoryMutationOperations.java deleted file mode 100644 index 7f38a27a5..000000000 --- a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/MemoryMutationOperations.java +++ /dev/null @@ -1,135 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.memory; - -import fr.inria.corese.core.next.data.api.model.Statement; -import fr.inria.corese.core.next.data.api.term.Resource; -import fr.inria.corese.core.next.storage.api.operations.MutationOperations; -import fr.inria.corese.core.next.storage.api.exception.ErrorCode; -import fr.inria.corese.core.next.storage.api.exception.StorageException; -import fr.inria.corese.core.next.storage.api.model.StatementPattern; - -import java.util.Objects; - -/** - * Mutation operations implementation for {@link MemoryStorageManager}. - */ -final class MemoryMutationOperations implements MutationOperations { - - private final InMemoryStatementStore adapter; - - /** - * Constructs a new MemoryMutationOperations. - * - * @param adapter the InMemoryStatementStore for data access (must not be null) - * @throws IllegalArgumentException if adapter is null - */ - public MemoryMutationOperations(InMemoryStatementStore adapter) { - if (adapter == null) throw new IllegalArgumentException("InMemoryStatementStore cannot be null"); - this.adapter = adapter; - } - - /** - * Creates an empty named graph in the memory store. - * - * @param graph the named graph identifier (must not be {@code null}) - * @return {@code true} if created, {@code false} if already present - * @throws NullPointerException if {@code graph} is {@code null} - */ - @Override - public boolean createGraph(Resource graph) { - return adapter.createGraph(graph); - } - - /** - * Drops a named graph and removes all of its contained statements. - * - * @param graph the named graph identifier (must not be {@code null}) - * @return {@code true} if dropped, {@code false} if absent - * @throws NullPointerException if {@code graph} is {@code null} - */ - @Override - public boolean dropGraph(Resource graph) { - return adapter.dropGraph(graph); - } - - /** - * Inserts a statement into the store. - * - * @param statement the statement to insert (must not be null) - * @return whether the store changed - * @throws NullPointerException if statement is null - * @throws StorageException if the insert operation fails - */ - @Override - public boolean add(Statement statement) throws StorageException { - Objects.requireNonNull(statement, "statement"); - try { - return adapter.add(statement); - } catch (Exception e) { - throw new StorageException(ErrorCode.MUTATION_FAILED, "Insert failed", e); - } - } - - /** - * Deletes a statement from the store. - * - * @param statement the statement to delete (must not be null) - * @return whether the store changed - * @throws NullPointerException if statement is null - * @throws StorageException if the delete operation fails - */ - @Override - public boolean remove(Statement statement) throws StorageException { - Objects.requireNonNull(statement, "statement"); - try { - return adapter.remove(statement); - } catch (Exception e) { - throw new StorageException(ErrorCode.MUTATION_FAILED, "Delete failed", e); - } - } - - /** - * Deletes all statements matching the given pattern. - * - * @param pattern statement pattern - * @return number of deleted statements - * @throws StorageException if the operation fails - */ - @Override - public long remove(StatementPattern pattern) throws StorageException { - Objects.requireNonNull(pattern, "pattern"); - try { - var toRemove = adapter.find( - pattern.getSubject(), - pattern.getPredicate(), - pattern.getObject(), - pattern.getContexts()); - long count = toRemove.size(); - toRemove.forEach(adapter::remove); - return count; - } catch (Exception e) { - throw new StorageException(ErrorCode.MUTATION_FAILED, "Delete failed", e); - } - } - - /** - * Clears statements from specified contexts. - * - * @param contexts contexts to clear; null/empty clears every graph and an - * explicit null element clears the default graph - * @return number of deleted statements - * @throws StorageException if the clear operation fails - */ - @Override - public long clear(Resource... contexts) throws StorageException { - try { - if (contexts != null && contexts.length > 0) { - return remove(StatementPattern.of(null, null, null, contexts)); - } - int before = adapter.size(); - adapter.clear(); - return (long) before - adapter.size(); - } catch (Exception e) { - throw new StorageException(ErrorCode.MUTATION_FAILED, "Clear failed", e); - } - } -} diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/MemoryQueryOperations.java b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/MemoryQueryOperations.java deleted file mode 100644 index 92c0ddaab..000000000 --- a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/MemoryQueryOperations.java +++ /dev/null @@ -1,79 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.memory; - -import fr.inria.corese.core.next.data.api.model.Statement; -import fr.inria.corese.core.next.storage.api.operations.QueryOperations; -import fr.inria.corese.core.next.storage.api.exception.ErrorCode; -import fr.inria.corese.core.next.storage.api.exception.StorageException; -import fr.inria.corese.core.next.storage.api.model.StatementPattern; - -import java.util.Objects; -import java.util.stream.Stream; - -/** - * Query operations implementation for {@link MemoryStorageManager}. - */ -final class MemoryQueryOperations implements QueryOperations { - - private final InMemoryStatementStore adapter; - - /** - * Constructs a new MemoryQueryOperations. - * - * @param adapter the InMemoryStatementStore for data access (must not be null) - * @throws NullPointerException if adapter is null - */ - public MemoryQueryOperations(InMemoryStatementStore adapter) { - this.adapter = Objects.requireNonNull(adapter, "adapter"); - } - - /** - * Queries for statements matching the given pattern. - * - * @param pattern the statement pattern to match (must not be null) - * @return a stream of matching statements (must be closed) - * @throws NullPointerException if pattern is null - * @throws StorageException if the query fails - */ - @Override - public Stream find(StatementPattern pattern) throws StorageException { - Objects.requireNonNull(pattern, "pattern"); - try { - return adapter.find( - pattern.getSubject(), - pattern.getPredicate(), - pattern.getObject(), - pattern.getContexts() - ).stream(); - } catch (Exception e) { - throw new StorageException(ErrorCode.QUERY_FAILED, "Query failed", e); - } - } - - /** - * Counts the number of statements matching the given pattern. - * - * @param pattern the statement pattern to match (must not be null) - * @return the count of matching statements - * @throws StorageException if the query fails - */ - @Override - public long count(StatementPattern pattern) throws StorageException { - try (Stream stream = find(pattern)) { - return stream.count(); - } - } - - /** - * Checks if any statement matches the given pattern. - * - * @param pattern the statement pattern to match (must not be null) - * @return true if at least one matching statement exists - * @throws StorageException if the query fails - */ - @Override - public boolean contains(StatementPattern pattern) throws StorageException { - try (Stream stream = find(pattern)) { - return stream.findAny().isPresent(); - } - } -} diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/MemoryStorageManager.java b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/MemoryStorageManager.java index 4cbbabb42..a55d2c001 100644 --- a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/MemoryStorageManager.java +++ b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/MemoryStorageManager.java @@ -1,129 +1,158 @@ package fr.inria.corese.core.next.storage.impl.memory; import fr.inria.corese.core.next.storage.api.StorageManager; +import fr.inria.corese.core.next.storage.api.config.StorageConfig; import fr.inria.corese.core.next.storage.api.lifecycle.StorageLifecycle; import fr.inria.corese.core.next.storage.api.operations.MetadataOperations; import fr.inria.corese.core.next.storage.api.operations.MutationOperations; import fr.inria.corese.core.next.storage.api.operations.QueryOperations; import fr.inria.corese.core.next.storage.api.transaction.TransactionManager; -import java.util.Objects; - /** - * In-memory {@link StorageManager} implementation for testing and small to - * medium datasets. + * Native indexed memory backend. Sessions share committed data, but own transactions. Transactions and lazy reads + * capture graph partitions, which are copied on the first mutation while shared. A read captures its version when + * {@code find()} is called, without materializing matches. Consumers must close returned streams, including after + * partial traversal, to release snapshot ownership. * - *

Transactions are local to the thread that starts them and use a private - * snapshot. A connection must not begin a transaction on one thread and - * commit it on another. Snapshot memory usage grows with the number of stored - * statements, so a persistent MVCC backend is preferable for very large - * datasets.

+ *

+ * Closing a session rolls back its active transaction. Closing the owning manager also shuts down the database. Already + * captured read streams retain their own partition ownership until they are closed or exhausted. + *

*/ public final class MemoryStorageManager implements StorageManager { + /** + * Session-scoped operation facade. + */ + private final IndexedStatementStore store; + /** + * Owning manager flag or transaction thread, according to the enclosing class. + */ + private final boolean owner; - private final InMemoryStatementStore adapter; - private final QueryOperations queryOps; - private final MutationOperations mutationOps; - private final MetadataOperations metadataOps; - private final TransactionManager txManager; - private final StorageLifecycle lifecycle; + /** + * Creates and initializes an owned backend. Normally use Storages.create(config). + * + * @param config storage configuration selecting this backend + */ + public MemoryStorageManager(final StorageConfig config) { + store = new IndexedStatementStore(); + owner = true; + store.lifecycle().initialize(config); + } /** - * Constructs a new MemoryStorageManager. + * Creates a non-owning session facade sharing the supplied operation store. * - * @param builder the builder with configuration + * @param store session-scoped operations sharing the owning database */ - private MemoryStorageManager(Builder builder) { - Objects.requireNonNull(builder, "Builder cannot be null"); - this.adapter = new InMemoryStatementStore(); - this.queryOps = new MemoryQueryOperations(adapter); - this.mutationOps = new MemoryMutationOperations(adapter); - this.metadataOps = new MemoryMetadataOperations(adapter); - this.txManager = new MemoryTransactionManager(adapter); - this.lifecycle = new MemoryLifecycleManager(adapter); + private MemoryStorageManager(final IndexedStatementStore store) { + this.store = store; + owner = false; } /** - * Constructs a scoped session of MemoryStorageManager sharing storage lifecycle. + * Builder for MemoryStorageManager instances. + */ + public static final class Builder { + private StorageConfig config = StorageConfig.memory(); + + public Builder config(StorageConfig config) { + this.config = config; + return this; + } + + public MemoryStorageManager build() { + return new MemoryStorageManager(config); + } + } + + /** + * Creates a builder for MemoryStorageManager. * - * @param adapter the session statement store - * @param lifecycle the shared storage lifecycle + * @return a new builder */ - private MemoryStorageManager(InMemoryStatementStore adapter, StorageLifecycle lifecycle) { - this.adapter = adapter; - this.queryOps = new MemoryQueryOperations(adapter); - this.mutationOps = new MemoryMutationOperations(adapter); - this.metadataOps = new MemoryMetadataOperations(adapter); - this.txManager = new MemoryTransactionManager(adapter); - this.lifecycle = lifecycle; + public static Builder builder() { + return new Builder(); } /** - * Returns a new connection session with an isolated transaction snapshot. + * {@inheritDoc} * - * @return a connection-scoped {@link StorageManager} + * @return a new session sharing the database */ @Override public StorageManager openSession() { - return new MemoryStorageManager(adapter.openSession(), lifecycle); + return new MemoryStorageManager(store.openSession()); } + /** + * {@inheritDoc} + * + * @return the session query operations + */ @Override public QueryOperations queries() { - return queryOps; + return store; } + /** + * {@inheritDoc} + * + * @return the session mutation operations + */ @Override public MutationOperations mutations() { - return mutationOps; + return store; } + /** + * {@inheritDoc} + * + * @return the session metadata operations + */ @Override public MetadataOperations metadata() { - return metadataOps; + return store; } + /** + * {@inheritDoc} + * + * @return the session transaction controller + */ @Override public TransactionManager transactions() { - return txManager; + return store; } + /** + * {@inheritDoc} + * + * @return the shared database lifecycle controller + */ @Override public StorageLifecycle lifecycle() { - return lifecycle; + return store.lifecycle(); } /** - * Returns a new builder for MemoryStorageManager. + * {@inheritDoc} * - * @return a new builder instance + * @return true when the session is open and the database is running */ - public static Builder builder() { - return new Builder(); + @Override + public boolean isOpen() { + return store.isOpen(); } /** - * Builder for {@link MemoryStorageManager}. + * {@inheritDoc} */ - public static final class Builder { - - - public Builder() { - // No configuration needed for MemoryStorageManager - } - - /** - * Builds a new MemoryStorageManager instance. - * - * @return a new MemoryStorageManager - */ - public MemoryStorageManager build() { - return new MemoryStorageManager(this); - } - } - @Override - public String toString() { - return "MemoryStorageManager{size=" + adapter.size() + "}"; + public synchronized void close() { + store.closeSession(); + if (owner && lifecycle().getState() == fr.inria.corese.core.next.storage.api.lifecycle.LifecycleState.RUNNING) { + lifecycle().shutdown(); + } } } diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/MemoryTransactionManager.java b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/MemoryTransactionManager.java deleted file mode 100644 index 8a3b6aa38..000000000 --- a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/MemoryTransactionManager.java +++ /dev/null @@ -1,133 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.memory; - -import fr.inria.corese.core.next.storage.api.transaction.IsolationLevel; -import fr.inria.corese.core.next.storage.api.transaction.Transaction; -import fr.inria.corese.core.next.storage.api.transaction.TransactionManager; -import fr.inria.corese.core.next.storage.api.transaction.TransactionState; - -import java.util.Set; -import java.util.UUID; - -/** - * Transaction manager for {@link MemoryStorageManager}. - * - *

Provides optimistic serializable transactions with private snapshots and conflict detection.

- */ -final class MemoryTransactionManager implements TransactionManager { - private final InMemoryStatementStore store; - - /** - * Constructs a new transaction manager for the specified memory store. - * - * @param store the underlying statement store - */ - MemoryTransactionManager(InMemoryStatementStore store) { - this.store = store; - } - - /** - * {@inheritDoc} - * - * @return {@code true} as in-memory storage supports transactions - */ - @Override - public boolean supportsTransactions() { - return true; - } - - /** - * Begins a new transaction with default isolation level ({@link IsolationLevel#SERIALIZABLE}). - * - * @return the created transaction handle - */ - @Override - public Transaction beginTransaction() { - return beginTransaction(IsolationLevel.SERIALIZABLE); - } - - /** - * Begins a new transaction with the specified isolation level. - * - * @param level the desired isolation level - * @return the created transaction handle - * @throws IllegalArgumentException if {@code level} is not supported - */ - @Override - public Transaction beginTransaction(IsolationLevel level) { - if (!getSupportedIsolationLevels().contains(level)) { - throw new IllegalArgumentException("Unsupported isolation level: " + level); - } - return new SnapshotTransaction(store.begin()); - } - - /** - * Returns the set of isolation levels supported by this manager. - * - * @return set containing {@link IsolationLevel#SERIALIZABLE} - */ - @Override - public Set getSupportedIsolationLevels() { - return Set.of(IsolationLevel.SERIALIZABLE); - } - - /** - * Snapshot-isolated transaction with thread-affinity and conflict detection. - */ - private final class SnapshotTransaction implements Transaction { - private final String id = UUID.randomUUID().toString(); - private final Thread owner = Thread.currentThread(); - private final long version; - private TransactionState state = TransactionState.ACTIVE; - - /** - * Constructs a new snapshot transaction. - * - * @param version the database version at snapshot creation - */ - SnapshotTransaction(long version) { - this.version = version; - } - - @Override - public String getId() { - return id; - } - - @Override - public void commit() { - requireActive(); - store.commit(version); - state = TransactionState.COMMITTED; - } - - @Override - public void rollback() { - requireActive(); - store.rollback(); - state = TransactionState.ROLLED_BACK; - } - - private void requireActive() { - if (Thread.currentThread() != owner || !isActive()) { - throw new IllegalStateException("Transaction is inactive or belongs to another thread"); - } - } - - @Override - public boolean isActive() { - return state == TransactionState.ACTIVE; - } - - @Override - public TransactionState getState() { - return state; - } - - @Override - public void close() { - if (isActive()) { - rollback(); - } - } - } -} diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/StagingTerms.java b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/StagingTerms.java new file mode 100644 index 000000000..cb5ee44ff --- /dev/null +++ b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/StagingTerms.java @@ -0,0 +1,123 @@ +package fr.inria.corese.core.next.storage.impl.memory; + +import fr.inria.corese.core.next.data.api.term.Value; + +import java.util.HashMap; + +/** + * Batch-private canonical dictionary, discarded before a partition is published. Each instance owns its mutable lookup + * probe and is confined to one batch writer. Serialization is outside this transient dictionary's contract; HashMap is + * used solely for canonical lookup and insertion. + */ +@SuppressWarnings({ "serial", "java:S1948", "java:S2160" }) +final class StagingTerms extends HashMap { + /** + * Mutable non-stored search probe owned exclusively by this staging dictionary. + */ + private final Lookup lookup = new Lookup(); + + /** + * Creates a batch-private canonicalization dictionary with an independent search probe. + */ + StagingTerms() { + } + + /** + * Creates a batch-private canonicalization dictionary with an independent search probe. + * + * @param expectedTerms bounded expected number of distinct staged terms + */ + StagingTerms(final int expectedTerms) { + super((expectedTerms * 4 + 2) / 3); + } + + /** + * Creates a batch-private canonicalization dictionary with an independent search probe. + * + * @param source source index or container; it must remain unchanged during copying or traversal + */ + StagingTerms(final StagingTerms source) { + super(source); + } + + /** + * Captures matching graph partitions immediately and returns a sequential lazy stream that must be closed. + * + *

+ * The private, non-stored probe is passed to HashMap.get(Object), which compares the search object's + * equals(storedKey). Stored keys always remain immutable TermKey instances. + * + * @param value non-null RDF value or entry value + * @param hash cached structural hash of the captured RDF value + * @return a sequential stream owning its read snapshot; the caller must close it + */ + @SuppressWarnings("java:S2175") + TermKey find(final Value value, final int hash) { + lookup.value = value; + lookup.hash = hash; + try { + return get(lookup); + } finally { + lookup.value = null; + } + } + + /** + * Reuses the canonical key for a captured term or records a new immutable key. + * + * @param value non-null RDF value or entry value + * @return the dictionary-owned canonical key + */ + TermKey intern(final Value value) { + final int hash = TermKey.structuralHash(value); + TermKey key = find(value, hash); + if (key == null) { + key = new TermKey(value, hash); + put(key, key); + } + return key; + } + + // HashMap.get invokes the search object's equals(storedKey). This probe is + // never inserted: every stored key is an immutable TermKey. Each private + // staging dictionary, including copies, owns its own probe. + /** + * Dictionary-private mutable search probe; it is never stored as a key and is not shared between writers. + */ + private static final class Lookup { + + /** Creates the writer-confined probe; its value is assigned only during lookup. */ + private Lookup() { + } + + /** + * Captured RDF value or transient lookup value, according to the enclosing class. + */ + private Value value; + /** + * Cached structural hash of the captured or transient lookup value. + */ + private int hash; + + /** + * {@inheritDoc} + * + * @return the cached structural hash + */ + @Override + public int hashCode() { + return hash; + } + + /** + * {@inheritDoc} + * + * @param other value or search object to compare with the stored key + * @return true when the other object satisfies this structural lookup contract + */ + @Override + public boolean equals(final Object other) { + return other == this || other instanceof TermKey key && key.matchesLookup(value, hash); + } + } +} diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/TermKey.java b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/TermKey.java new file mode 100644 index 000000000..5971e5da0 --- /dev/null +++ b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/TermKey.java @@ -0,0 +1,211 @@ +package fr.inria.corese.core.next.storage.impl.memory; + +import fr.inria.corese.core.next.data.api.term.BNode; +import fr.inria.corese.core.next.data.api.term.IRI; +import fr.inria.corese.core.next.data.api.term.Literal; +import fr.inria.corese.core.next.data.api.term.SimpleIRI; +import fr.inria.corese.core.next.data.api.term.Triple; +import fr.inria.corese.core.next.data.api.term.Value; + +/** + * Captured RDF identity with a cached structural hash for graph-local index lookup. + * Keys do not defensively copy values: the store captures supported RDF identities + * before indexing, and those identities must remain stable while the key is retained. + * Structural copies share key instances. The existing temporal-value accessors follow + * the native data API contract and are outside the container copy-on-write protocol. + */ +final class TermKey { + // String concatenation hashes without constructing an IRI string. Shared globally, + // never retained per term; lengths count UTF-16 units just like String.hashCode(). + /** + * Shared factors for UTF-16-compatible IRI string hashing. + */ + private static final int[] POWERS_OF_31 = powersOf31(); + /** + * Captured RDF value or transient lookup value, according to the enclosing class. + */ + private final Value value; + /** + * Cached structural hash of the captured or transient lookup value. + */ + private final int hash; + + /** + * Captures an RDF value and its structural hash for immutable index lookup. + * + * @param value non-null captured RDF value with stable lexical identity + */ + TermKey(final Value value) { + this(value, structuralHash(value)); + } + + /** + * Captures an RDF value and its structural hash for immutable index lookup. + * + * @param value non-null captured RDF value with stable lexical identity + * @param hash cached structural hash of the captured RDF value + */ + TermKey(final Value value, final int hash) { + this.value = value; + this.hash = hash; + } + + /** + * A null search component remains a wildcard, never an indexed term. + * + * @param value non-null captured RDF value with stable lexical identity + * @return a key for a non-null value, or null for a wildcard + */ + static TermKey nullable(final Value value) { + return value == null ? null : new TermKey(value); + } + + /** + * Reads an immutable RDF value or a value held in the container representation. + * + * @return the captured RDF value + */ + Value value() { + return value; + } + + /** + * Computes an RDF structural hash without allocating a temporary search key. + * + * @param value non-null captured RDF value with stable lexical identity + * @return the RDF structural hash + * @throws IllegalArgumentException if a capacity, term type or requested isolation level is unsupported + */ + static int structuralHash(final Value value) { + if (value instanceof IRI iri) { + return iriHash(iri); + } + if (value instanceof BNode blank) { + return blank.getID().hashCode(); + } + if (value instanceof Literal literal) { + final int result = 31 * literal.getLabel().hashCode() + iriHash(literal.getDatatype()); + return 31 * result + literal.getLanguage().hashCode(); + } + if (value instanceof Triple triple) { + final int result = 31 * structuralHash(triple.subject()) + structuralHash(triple.predicate()); + return 31 * result + structuralHash(triple.object()); + } + throw new IllegalArgumentException("Unsupported RDF term: " + value.getClass().getName()); + } + + /** + * Builds the shared UTF-16 length factors used for IRI string hashing. + * + * @return shared length factors for UTF-16 IRI hashing + */ + private static int[] powersOf31() { + final int[] powers = new int[32]; + powers[0] = 1; + for (int i = 1; i < powers.length; i++) { + powers[i] = 31 * powers[i - 1]; + } + return powers; + } + + /** + * Computes an IRI hash compatible with its full lexical string without concatenating native components. + * + * @param iri captured IRI whose lexical hash is required + * @return the hash of the IRI lexical string + */ + private static int iriHash(final IRI iri) { + if (!(iri instanceof SimpleIRI)) { + return iri.stringValue().hashCode(); + } + final String local = iri.getLocalName(); + final int length = local.length(); + if (length == 0) { + return iri.getNamespace().hashCode(); + } + int factor; + if (length < POWERS_OF_31.length) { + factor = POWERS_OF_31[length]; + } else { + // Exponentiation by squaring also handles unusually long local names. + factor = 1; + int power = 31; + for (int remaining = length; remaining != 0; remaining >>>= 1) { + if ((remaining & 1) != 0) { + factor *= power; + } + power *= power; + } + } + return factor * iri.getNamespace().hashCode() + local.hashCode(); + } + + /** + * {@inheritDoc} + * + * @return the cached structural hash + */ + @Override + public int hashCode() { + return hash; + } + + /** + * Compare a singleton's inline RDF value without allocating a second key. + * + * @param other value or search object to compare with the stored key + * @return true when the inline RDF value matches this key + */ + + boolean matches(final Value other) { + return value == other || hash == structuralHash(other) && sameValue(value, other); + } + + /** + * Lookup direction matches candidate.equals(storedKey), without a candidate object. + * + * @param candidate search key + * @param candidateHash structural hash of the search value + * @return true when the supplied hash and search value match this stored key + */ + + boolean matchesLookup(final Value candidate, final int candidateHash) { + return hash == candidateHash && sameValue(candidate, value); + } + + /** + * {@inheritDoc} + * + * @param other value or search object to compare with the stored key + * @return true when the other object satisfies this structural lookup contract + */ + @Override + public boolean equals(final Object other) { + if (this == other) { + return true; + } + if (!(other instanceof TermKey key)) { + return false; + } + return sameValue(value, key.value); + } + + /** + * Compares RDF lexical identities, including native IRIs with different namespace splits. + * + * @param value non-null captured RDF value with stable lexical identity + * @param other value or search object to compare with the stored key + * @return true when both RDF lexical identities are equal + */ + private static boolean sameValue(final Value value, final Value other) { + if (value == other) { + return true; + } + if (value instanceof SimpleIRI left && other instanceof SimpleIRI right + && left.getNamespace().equals(right.getNamespace())) { + return left.getLocalName().equals(right.getLocalName()); + } + // Different namespace/localName splits may still denote the same RDF IRI. + return value.equals(other); + } +} diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/TripleIndex.java b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/TripleIndex.java new file mode 100644 index 000000000..7cd41b0ef --- /dev/null +++ b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/TripleIndex.java @@ -0,0 +1,1049 @@ +package fr.inria.corese.core.next.storage.impl.memory; + +import fr.inria.corese.core.next.data.api.model.Statement; +import fr.inria.corese.core.next.data.api.term.Resource; +import fr.inria.corese.core.next.data.api.term.Value; +import fr.inria.corese.core.next.storage.api.model.StatementPattern; + +import java.util.Collection; +import java.util.List; +import java.util.Map; +import java.util.function.BiConsumer; +import java.util.function.Predicate; + +/** + * One insertion-ordered SPO, POS or OSP permutation within a graph partition. + * + *

+ * The dense root table maps the first term to a compact branch keyed by the second term. A branch value is either a + * single immutable statement or a compact leaf keyed by the third term. Singleton third keys are derived from the + * statement when needed. All three permutations share immutable statements and canonical {@link TermKey} instances; + * root branches retain their canonical first key so it can be reused across term roles without a persistent dictionary. + * + *

+ * Batch ingestion privately retains {@link StagingTerms} while building SPO and the secondary permutations. + * {@link #finishStaging()} discards that dictionary before the completed partition is published. The source is read + * once by the store; rebuilding secondary indexes traverses the staged SPO, not the source. + * + *

+ * {@link #copy()} clones root, branch and multivalued leaf buffers directly, preserving representation, holes and + * order. Only immutable terms and statements are shared. Mutations require an exclusively owned partition; cursors + * require their caller to retain an unchanged partition until traversal ends. + */ +final class TripleIndex { + /** + * Component ordering for one graph-local triple permutation. + */ + enum Order { + /** + * Subject/predicate/object permutation. + */ + SPO, + /** + * Predicate/object/subject permutation. + */ + POS, + /** + * Object/subject/predicate permutation. + */ + OSP; + + /** + * Selects the third component of this permutation. + * + * @param s captured statement to index + * @return the third permutation component, or null for a wildcard + */ + private Value third(final Statement s) { + return switch (this) { + case SPO -> s.getObject(); + case POS -> s.getSubject(); + case OSP -> s.getPredicate(); + }; + } + + /** + * Selects the first component of this permutation. + * + * @param p captured search pattern + * @return the first permutation component, or null for a wildcard + */ + private Value first(final StatementPattern p) { + return switch (this) { + case SPO -> p.getSubject(); + case POS -> p.getPredicate(); + case OSP -> p.getObject(); + }; + } + + /** + * Selects the second component of this permutation. + * + * @param p captured search pattern + * @return the second permutation component, or null for a wildcard + */ + private Value second(final StatementPattern p) { + return switch (this) { + case SPO -> p.getPredicate(); + case POS -> p.getObject(); + case OSP -> p.getSubject(); + }; + } + + /** + * Selects the third component of this permutation. + * + * @param p captured search pattern + * @return the third permutation component, or null for a wildcard + */ + private Value third(final StatementPattern p) { + return switch (this) { + case SPO -> p.getObject(); + case POS -> p.getSubject(); + case OSP -> p.getPredicate(); + }; + } + } + + /** + * Component order of this triple permutation. + */ + private final Order order; + /** + * Insertion-ordered first-component branches. + */ + private final Map roots; + // Bulk-only canonicalization cache, discarded before publishing a partition. + /** + * Batch-private canonical dictionary, cleared before publication. + */ + private StagingTerms stagingTerms; + /** + * Number of live entries or statements owned by this container. + */ + private long size; + + /** + * Creates an empty permutation or copies the source permutation containers. + * + * @param order SPO, POS or OSP component order + */ + TripleIndex(final Order order) { + this.order = order; + roots = new DenseTermTable<>(); + } + + /** + * Creates a private SPO index with a bounded initial canonicalization reservation. + * + * @param expectedTerms bounded expected number of distinct staged terms + * @return an unpublished SPO index + */ + static TripleIndex forStaging(final int expectedTerms) { + final TripleIndex index = new TripleIndex(Order.SPO); + if (expectedTerms > 0) { + index.stagingTerms = new StagingTerms(expectedTerms); + } + return index; + } + + /** + * Creates an empty permutation or copies the source permutation containers. + * + * @param source source index or container; it must remain unchanged during copying or traversal + */ + private TripleIndex(final TripleIndex source) { + order = source.order; + roots = new DenseTermTable<>((DenseTermTable) source.roots, Branch::new); + size = source.size; + if (source.stagingTerms != null) { + stagingTerms = new StagingTerms(source.stagingTerms); + } + } + + /** + * Adds a statement or keyed value if it is absent. + * + * @param s captured statement to index + * @param context canonical graph identifier; null denotes the default graph + * @return true when an absent entry was added, false when it was already present + */ + boolean add(final Statement s, final Resource context) { + if (order != Order.SPO) { + return add(s, new TermKey(s.getSubject()), new TermKey(s.getPredicate()), new TermKey(s.getObject())); + } + if (stagingTerms == null) { + stagingTerms = new StagingTerms(); + // Seed stored keys before deriving omitted singleton keys: cross-role + // aliases must reuse the existing subject/predicate/object key objects. + roots.forEach((first, branch) -> stagingTerms.putIfAbsent(first, first)); + roots.values() + .forEach(branch -> branch.forEach((second, entry) -> stagingTerms.putIfAbsent(second, second))); + roots.values().forEach(branch -> branch.forEach((second, entry) -> { + if (entry instanceof Leaf leaf) { + leaf.forEach((third, statement) -> stagingTerms.putIfAbsent(third, third)); + } + })); + roots.values().forEach(branch -> branch.forEach((second, entry) -> { + if (!(entry instanceof Leaf)) { + stagedKey(order.third((Statement) entry)); + } + })); + } + final TermKey subject = stagedKey(s.getSubject()); + final TermKey predicate = stagedKey(s.getPredicate()); + final TermKey object = stagedKey(s.getObject()); + return add(IndexedStatementStore.sharedStatement(s, subject, predicate, object, context), subject, predicate, + object); + } + + /** + * Interns a captured RDF value in the batch-private dictionary. + * + * @param value non-null RDF value or entry value + * @return the staging dictionary-owned canonical key + */ + private TermKey stagedKey(final Value value) { + return stagingTerms.intern(value); + } + + /** + * Reuses a canonical key from staging or role roots, creating one only when it is absent. + * + * @param value non-null RDF value or entry value + * @return an existing canonical key, or a newly captured immutable key + */ + private TermKey canonicalKey(final Value value) { + final int hash = TermKey.structuralHash(value); + TermKey key = stagingTerms == null ? null : stagingTerms.find(value, hash); + if (key == null) { + key = existingKey(value, hash); + } + return key == null ? new TermKey(value, hash) : key; + } + + /** + * Looks up a canonical root key using an RDF value and its structural hash. + * + * @param value non-null RDF value or entry value + * @param hash cached structural hash of the captured RDF value + * @return the canonical root key, or null if absent + */ + TermKey existingKey(final Value value, final int hash) { + final Branch branch = ((DenseTermTable) roots).get(value, hash); + return branch == null ? null : branch.key; + } + + /** Releases the batch-only dictionary before this index becomes visible to readers. */ + void finishStaging() { + stagingTerms = null; + } + + /** + * Adds a statement or keyed value if it is absent. + * + * @param s captured statement to index + * @param subject canonical subject key, or null for a wildcard search + * @param predicate canonical predicate key; null denotes a wildcard for search operations + * @param object canonical object key, or null for a wildcard search + * @return true when an absent entry was added, false when it was already present + */ + boolean add(final Statement s, final TermKey subject, final TermKey predicate, final TermKey object) { + return add(s, subject, predicate, object, this); + } + + /** + * Adds a statement or keyed value if it is absent. + * + * @param s captured statement to index + * @param subject canonical subject key, or null for a wildcard search + * @param predicate canonical predicate key; null denotes a wildcard for search operations + * @param object canonical object key, or null for a wildcard search + * @param thirdKeys permutation providing canonical keys for omitted singleton values + * @return true when an absent entry was added, false when it was already present + */ + boolean add(final Statement s, final TermKey subject, final TermKey predicate, final TermKey object, + final TripleIndex thirdKeys) { + final TermKey first = first(subject, predicate, object); + final TermKey second = second(subject, predicate, object); + final TermKey third = third(subject, predicate, object); + final Branch branch = roots.computeIfAbsent(first, Branch::new); + final Object entry = branch.get(second); + if (entry == null) { + branch.add(second, s); + } else if (entry instanceof Leaf leaf) { + if (!leaf.add(third, s)) { + return false; + } + } else { + final Statement singleton = (Statement) entry; + if (third.matches(order.third(singleton))) { + return false; + } + final Leaf leaf = new Leaf(); + leaf.add(thirdKeys.canonicalKey(order.third(singleton)), singleton); + leaf.add(third, s); + branch.replace(second, leaf); + } + size++; + return true; + } + + /** + * Selects the first component of this permutation. + * + * @param subject canonical subject key, or null for a wildcard search + * @param predicate canonical predicate key; null denotes a wildcard for search operations + * @param object canonical object key, or null for a wildcard search + * @return the first permutation component, or null for a wildcard + */ + private TermKey first(final TermKey subject, final TermKey predicate, final TermKey object) { + return switch (order) { + case SPO -> subject; + case POS -> predicate; + case OSP -> object; + }; + } + + /** + * Selects the second component of this permutation. + * + * @param subject canonical subject key, or null for a wildcard search + * @param predicate canonical predicate key; null denotes a wildcard for search operations + * @param object canonical object key, or null for a wildcard search + * @return the second permutation component, or null for a wildcard + */ + private TermKey second(final TermKey subject, final TermKey predicate, final TermKey object) { + return switch (order) { + case SPO -> predicate; + case POS -> object; + case OSP -> subject; + }; + } + + /** + * Selects the third component of this permutation. + * + * @param subject canonical subject key, or null for a wildcard search + * @param predicate canonical predicate key; null denotes a wildcard for search operations + * @param object canonical object key, or null for a wildcard search + * @return the third permutation component, or null for a wildcard + */ + private TermKey third(final TermKey subject, final TermKey predicate, final TermKey object) { + return switch (order) { + case SPO -> object; + case POS -> subject; + case OSP -> predicate; + }; + } + + /** + * Removes matching content while preserving the order of survivors. + * + * @param subject canonical subject key, or null for a wildcard search + * @param predicate canonical predicate key; null denotes a wildcard for search operations + * @param object canonical object key, or null for a wildcard search + * @return true when an entry was removed, false when it was absent + */ + boolean remove(final TermKey subject, final TermKey predicate, final TermKey object) { + final TermKey first = first(subject, predicate, object); + final TermKey second = second(subject, predicate, object); + final Branch branch = roots.get(first); + if (branch == null) { + return false; + } + final Object entry = branch.get(second); + if (entry == null) { + return false; + } + final TermKey third = third(subject, predicate, object); + if (entry instanceof Leaf leaf) { + if (!leaf.remove(third)) { + return false; + } + if (leaf.size() == 1) { + branch.replace(second, leaf.singletonValue()); + } + } else { + if (!third.matches(order.third((Statement) entry))) { + return false; + } + branch.remove(second); + } + size--; + if (branch.isEmpty()) { + roots.remove(first); + } + return true; + } + + /** + * Broad pattern removal: stable filtering, with one rebuild per affected dense table. + * + * @param subject canonical subject key, or null for a wildcard search + * @param predicate canonical predicate key; null denotes a wildcard for search operations + * @param object canonical object key, or null for a wildcard search + * @return number of matching statements removed + */ + + long removeMatches(final TermKey subject, final TermKey predicate, final TermKey object) { + final TermKey first = first(subject, predicate, object); + final TermKey second = second(subject, predicate, object); + final TermKey third = third(subject, predicate, object); + final long[] removed = { 0 }; + if (first != null) { + final Branch branch = roots.get(first); + if (branch != null) { + removed[0] = removeMatches(branch, second, third); + if (branch.isEmpty()) { + roots.remove(first); + } + } + } else { + ((DenseTermTable) roots).removeIf(branch -> { + removed[0] += removeMatches(branch, second, third); + return branch.isEmpty(); + }); + } + size -= removed[0]; + return removed[0]; + } + + /** + * Removes selected index content with stable filtering and unchanged compaction rules. + * + * @param branch owned or retained branch appropriate to this operation + * @param second second-component key; null selects all branch values + * @param third third-component key; null selects all leaf values + * @return number of matching statements removed + */ + private long removeMatches(final Branch branch, final TermKey second, final TermKey third) { + if (second != null) { + return removeSelectedEntry(branch, second, third); + } + final long[] removed = { 0 }; + branch.removeIf(entry -> { + if (third == null) { + removed[0] += entryCount(entry, null); + return true; + } + if (entry instanceof Leaf leaf) { + if (leaf.remove(third)) { + removed[0]++; + } + return leaf.isEmpty(); + } + if (!third.matches(order.third((Statement) entry))) { + return false; + } + removed[0]++; + return true; + }); + // Demote after filtering: replacing values is safe in this stable traversal. + if (third != null) { + branch.forEach((key, entry) -> { + if (entry instanceof Leaf leaf && leaf.size() == 1) { + branch.replace(key, leaf.singletonValue()); + } + }); + } + return removed[0]; + } + + /** + * Removes matching content under one selected second-component key. + * + * @param branch owned or retained branch appropriate to this operation + * @param second second-component key; null selects all branch values + * @param third third-component key; null selects all leaf values + * @return number of matching statements removed under the selected key + */ + private long removeSelectedEntry(final Branch branch, final TermKey second, final TermKey third) { + final Object entry = branch.get(second); + if (entry == null) { + return 0; + } + if (third == null) { + final long removed = entryCount(entry, null); + branch.remove(second); + return removed; + } + if (entry instanceof Leaf leaf) { + if (!leaf.remove(third)) { + return 0; + } + if (leaf.isEmpty()) { + branch.remove(second); + } else if (leaf.size() == 1) { + branch.replace(second, leaf.singletonValue()); + } + return 1; + } + if (!third.matches(order.third((Statement) entry))) { + return 0; + } + branch.remove(second); + return 1; + } + + /** + * Returns the number of live entries. + * + * @return live entry count + */ + long size() { + return size; + } + + /** + * Returns a materialized, ordered collection of the immutable first-component values. + * + * @return immutable first-component values in encounter order + */ + Collection firstKeys() { + return roots.keySet().stream().map(TermKey::value).toList(); + } + + /** + * Stops as soon as visitor returns true; visits only matching index leaves. + * + * @param pattern captured search pattern + * @param visitor non-mutating callback; true requests an early stop for predicate visitors + * @return true when a matching value caused the visitor to request an early stop + */ + + boolean visit(final StatementPattern pattern, final Predicate visitor) { + final Value second = order.second(pattern); + final Value third = order.third(pattern); + + final TermKey secondKey = second == null ? null : new TermKey(second); + final TermKey thirdKey = third == null ? null : new TermKey(third); + for (var branch : selected(roots, order.first(pattern))) { + if (branch.visitMatching(secondKey, + entry -> entry instanceof Leaf leaf ? leaf.visitMatching(thirdKey, visitor) + : (thirdKey == null || thirdKey.matches(order.third((Statement) entry))) + && visitor.test((Statement) entry))) { + return true; + } + } + return false; + } + + /** + * Search keys are prepared by the caller once, then reused across graph partitions. + * + * @param subject canonical subject key, or null for a wildcard search + * @param predicate canonical predicate key; null denotes a wildcard for search operations + * @param object canonical object key, or null for a wildcard search + * @param visitor non-mutating callback; true requests an early stop for predicate visitors + * @return true when a matching value caused the visitor to request an early stop + */ + + boolean visit(final TermKey subject, final TermKey predicate, final TermKey object, + final Predicate visitor) { + final TermKey secondKey = second(subject, predicate, object); + final TermKey thirdKey = third(subject, predicate, object); + + for (var branch : selected(roots, first(subject, predicate, object))) { + if (branch.visitMatching(secondKey, + entry -> entry instanceof Leaf leaf ? leaf.visitMatching(thirdKey, visitor) + : (thirdKey == null || thirdKey.matches(order.third((Statement) entry))) + && visitor.test((Statement) entry))) { + return true; + } + } + return false; + } + + /** + * Selects the complete ordered value view or the value of one key. + * + * @param map unchanged ordered map to select from + * @param key search or insertion key; null denotes a wildcard only for pattern traversal + * @param selected entry value type + * @return the complete ordered value view or an immutable singleton selection + */ + private static Collection selected(final Map map, final Value key) { + if (key == null) { + return map.values(); + } + final T value = map.get(new TermKey(key)); + return value == null ? List.of() : List.of(value); + } + + /** + * Selects the complete ordered value view or the value of one key. + * + * @param map unchanged ordered map to select from + * @param key search or insertion key; null denotes a wildcard only for pattern traversal + * @param selected entry value type + * @return the complete ordered value view or an immutable singleton selection + */ + private static Collection selected(final Map map, final TermKey key) { + if (key == null) { + return map.values(); + } + final T value = map.get(key); + return value == null ? List.of() : List.of(value); + } + + /** + * Copies all mutable index containers, sharing only immutable keys and statements. + * + * @return an independently owned copy of the mutable containers + */ + TripleIndex copy() { + return new TripleIndex(this); + } + + /** + * Caller retains the partition for the entire cursor lifetime. No per-prefix iterators. + * + * @param subject canonical subject key, or null for a wildcard search + * @param predicate canonical predicate key; null denotes a wildcard for search operations + * @param object canonical object key, or null for a wildcard search + * @return a single-consumer positional cursor + */ + Cursor cursor(final TermKey subject, final TermKey predicate, final TermKey object) { + return new Cursor(first(subject, predicate, object), second(subject, predicate, object), + third(subject, predicate, object)); + } + + /** + * Single-consumer positional cursor over an unchanged partition retained by its caller. + */ + final class Cursor { + /** + * Captured first-component search key; null denotes a wildcard. + */ + private final TermKey first; + /** + * Captured second-component search key; null denotes a wildcard. + */ + private final TermKey second; + /** + * Captured third-component search key; null denotes a wildcard. + */ + private final TermKey third; + /** + * Next root offset to inspect in the retained table. + */ + private int rootPosition; + /** + * Next branch offset to inspect. + */ + private int branchPosition; + /** + * Next multivalued leaf offset to inspect. + */ + private int leafPosition; + /** + * Whether the explicitly selected root has already been visited. + */ + private boolean selectedRoot; + /** + * Current unchanged branch retained by the enclosing snapshot. + */ + private Branch branch; + /** + * Current unchanged multivalued leaf. + */ + private Leaf leaf; + + /** + * Captures the three search keys for unchanged positional traversal. + * + * @param first first-component key; null selects all roots + * @param second second-component key; null selects all branch values + * @param third third-component key; null selects all leaf values + */ + private Cursor(final TermKey first, final TermKey second, final TermKey third) { + this.first = first; + this.second = second; + this.third = third; + } + + /** + * Internal cursor: null marks exhaustion. The public Spliterator supplies Iterator semantics. + * + * @return the next statement, or null at exhaustion for the internal index cursor + */ + Statement next() { + for (;;) { + if (leaf != null) { + final Statement result = nextLeafStatement(); + if (result != null) { + return result; + } + } + if (branch != null) { + final Statement result = nextBranchStatement(); + if (result != null) { + return result; + } + continue; + } + if (!selectNextRoot()) { + return null; + } + branchPosition = 0; + } + } + + /** + * Reads the next match from the current leaf and drops that traversal reference on exhaustion. + * + * @return the next matching leaf statement, or null when the leaf is exhausted + */ + private Statement nextLeafStatement() { + if (third != null) { + final Statement result = leaf.get(third); + leaf = null; + if (result != null) { + return result; + } + } else { + final int position = leaf.nextIndex(leafPosition); + if (position >= 0) { + leafPosition = position + 1; + return leaf.valueAt(position); + } + leaf = null; + } + return null; + } + + /** + * Reads a branch match or selects a multivalued leaf for subsequent traversal. + * + * @return a matching singleton statement, or null when traversal must continue + */ + private Statement nextBranchStatement() { + final Object entry = nextBranchEntry(); + if (entry instanceof Leaf values) { + leaf = values; + leafPosition = 0; + return null; + } + if (entry != null && (third == null || third.matches(order.third((Statement) entry)))) { + return (Statement) entry; + } + return null; + } + + /** + * Reads a selected branch value or advances its positional traversal. + * + * @return the selected branch entry, or null when no entry remains + */ + private Object nextBranchEntry() { + Object entry; + if (second != null) { + entry = branch.get(second); + branch = null; + } else { + final int position = branch.nextIndex(branchPosition); + if (position < 0) { + branch = null; + return null; + } + branchPosition = position + 1; + entry = branch.valueAt(position); + } + return entry; + } + + /** + * Selects the next root once, preserving its encounter order. + * + * @return true when a root was selected, false at exhaustion + */ + private boolean selectNextRoot() { + if (first != null) { + if (selectedRoot) { + return false; + } + selectedRoot = true; + branch = roots.get(first); + if (branch == null) { + return false; + } + } else { + var table = (DenseTermTable) roots; + final int position = table.nextIndex(rootPosition); + if (position < 0) { + return false; + } + rootPosition = position + 1; + branch = table.valueAt(position); + } + return true; + } + + } + + /** + * Counts matching prefixes without visiting every statement in a dense leaf. + * + * @param pattern captured search pattern + * @return matching statement count + */ + + long count(final StatementPattern pattern) { + if (order.first(pattern) == null && order.second(pattern) == null && order.third(pattern) == null) { + return size; + } + long count = 0; + final Value third = order.third(pattern); + final TermKey thirdKey = third == null ? null : new TermKey(third); + for (var branch : selected(roots, order.first(pattern))) { + final Value second = order.second(pattern); + if (second == null) { + count += branch.sum(entry -> entryCount(entry, thirdKey)); + } else { + final Object entry = branch.get(new TermKey(second)); + if (entry != null) { + count += entryCount(entry, thirdKey); + } + } + } + return count; + } + + /** + * Counts matching statements in the captured or session-visible state. + * + * @param subject canonical subject key, or null for a wildcard search + * @param predicate canonical predicate key; null denotes a wildcard for search operations + * @param object canonical object key, or null for a wildcard search + * @return matching statement count + */ + long count(final TermKey subject, final TermKey predicate, final TermKey object) { + if (subject == null && predicate == null && object == null) { + return size; + } + long count = 0; + final TermKey secondKey = second(subject, predicate, object); + final TermKey thirdKey = third(subject, predicate, object); + + for (var branch : selected(roots, first(subject, predicate, object))) { + if (secondKey == null) { + count += branch.sum(entry -> entryCount(entry, thirdKey)); + } else { + final Object entry = branch.get(secondKey); + if (entry != null) { + count += entryCount(entry, thirdKey); + } + } + } + return count; + } + + /** + * Counts matches in a singleton statement or a multivalued leaf. + * + * @param entry immutable singleton statement or compact multivalued leaf + * @param third third-component key; null selects all leaf values + * @return matching leaf cardinality + */ + private long entryCount(final Object entry, final TermKey third) { + if (entry instanceof Leaf leaf) { + if (third == null) { + return leaf.size(); + } + return leaf.get(third) == null ? 0 : 1; + } + return third == null || third.matches(order.third((Statement) entry)) ? 1 : 0; + } + + /** Root keys are also addressable as values for cross-permutation canonicalization. */ + private static final class Branch extends CompactTermMap { + /** + * Canonical first-component key retained by this root branch. + */ + private final TermKey key; + + /** + * Creates a root branch retaining its canonical key or copies its nested containers. + * + * @param key search or insertion key; null denotes a wildcard only for pattern traversal + */ + Branch(final TermKey key) { + this.key = key; + } + + /** + * Creates a root branch retaining its canonical key or copies its nested containers. + * + * @param source source index or container; it must remain unchanged during copying or traversal + */ + Branch(final Branch source) { + super(source, entry -> entry instanceof Leaf leaf ? leaf.copy() : entry); + key = source.key; + } + } + + /** Multi-statement leaf only; singleton statements live directly in their branch. */ + private static final class Leaf extends CompactTermMap { + /** + * Creates an empty multivalued leaf or copies the source leaf container. + */ + Leaf() { + } + + /** + * Creates an empty multivalued leaf or copies the source leaf container. + * + * @param source source index or container; it must remain unchanged during copying or traversal + */ + Leaf(final Leaf source) { + super(source); + } + + /** + * Copies owned mutable containers while sharing immutable keys and statements. + * + * @return an independently owned copy of the mutable containers + */ + private Leaf copy() { + return new Leaf(this); + } + } + + /** + * Receives immutable SPO statements and their canonical term keys during index construction. + */ + @FunctionalInterface + private interface EntryVisitor { + /** + * Receives one immutable statement and its canonical SPO keys. + * + * @param subject canonical subject key, or null for a wildcard search + * @param predicate canonical predicate key; null denotes a wildcard for search operations + * @param object canonical object key, or null for a wildcard search + * @param statement statement whose RDF identities have been captured + */ + void accept(final TermKey subject, final TermKey predicate, final TermKey object, final Statement statement); + } + + /** + * Visits canonical SPO entries while constructing secondary permutations. + * + * @param visitor non-mutating callback; true requests an early stop for predicate visitors + * @throws IllegalStateException if the required representation, ownership or lifecycle state is not satisfied + */ + private void visitEntries(final EntryVisitor visitor) { + if (order != Order.SPO) { + throw new IllegalStateException("Bulk source must be SPO"); + } + final EntryTraversal traversal = new EntryTraversal(this, visitor); + roots.forEach(traversal::root); + } + + /** One call-local traversal, avoiding a capturing callback per prefix/leaf. */ + private static final class EntryTraversal { + /** + * SPO index supplying statements and canonical keys. + */ + private final TripleIndex source; + /** + * Batch-local canonical-entry callback. + */ + private final EntryVisitor visitor; + /** + * Callback reused across predicate branches. + */ + private final BiConsumer branches = this::branch; + /** + * Callback reused across multivalued object leaves. + */ + private final BiConsumer leaves = this::leaf; + /** + * Canonical traversal subject or current batch subject, according to the enclosing traversal. + */ + private TermKey subject; + /** + * Canonical traversal predicate or current batch predicate, according to the enclosing traversal. + */ + private TermKey predicate; + + /** + * Creates call-local SPO traversal callbacks for the source and visitor. + * + * @param source source index or container; it must remain unchanged during copying or traversal + * @param visitor non-mutating callback; true requests an early stop for predicate visitors + */ + private EntryTraversal(final TripleIndex source, final EntryVisitor visitor) { + this.source = source; + this.visitor = visitor; + } + + /** + * Traverses one SPO root using call-local subject and predicate positions. + * + * @param subject canonical subject key, or null for a wildcard search + * @param branch owned or retained branch appropriate to this operation + */ + private void root(final TermKey subject, final Branch branch) { + this.subject = subject; + branch.forEach(branches); + } + + /** + * Traverses one predicate branch, recovering an omitted singleton object key if necessary. + * + * @param predicate canonical predicate key; null denotes a wildcard for search operations + * @param entry immutable singleton statement or compact multivalued leaf + */ + private void branch(final TermKey predicate, final Object entry) { + this.predicate = predicate; + if (entry instanceof Leaf leaf) { + leaf.forEach(leaves); + } else { + final Statement singleton = (Statement) entry; + visitor.accept(subject, predicate, source.canonicalKey(singleton.getObject()), singleton); + } + } + + /** + * Forwards one object-keyed statement with the current subject and predicate keys. + * + * @param object canonical object key, or null for a wildcard search + * @param statement statement whose RDF identities have been captured + */ + private void leaf(final TermKey object, final Statement statement) { + visitor.accept(subject, predicate, object, statement); + } + } + + /** + * Reconstruct omitted singleton keys once while building both secondary permutations. + * + * @param source source index or container; it must remain unchanged during copying or traversal + * @return the completed POS and OSP permutations + */ + static SecondaryIndexes buildSecondaryIndexes(final TripleIndex source) { + final TripleIndex pos = new TripleIndex(Order.POS); + final TripleIndex osp = new TripleIndex(Order.OSP); + source.visitEntries((subject, predicate, object, statement) -> { + pos.add(statement, subject, predicate, object, source); + osp.add(statement, subject, predicate, object, source); + }); + return new SecondaryIndexes(pos, osp); + } + + /** + * Completed secondary permutations sharing the SPO statements and canonical keys. + * + * @param pos completed POS permutation + * @param osp completed OSP permutation + */ + record SecondaryIndexes(/** + * Predicate/object/subject permutation of this graph. + */ + TripleIndex pos, /** + * Object/subject/predicate permutation of this graph. + */ + TripleIndex osp) { + } +} diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/package-info.java b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/package-info.java index 86252a384..4daf92a5d 100644 --- a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/package-info.java +++ b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/package-info.java @@ -1,7 +1,20 @@ /** - * In-memory storage implementation. + * Native in-memory RDF storage with insertion-ordered SPO, POS and OSP indexes. * - *

Stores RDF statements in heap memory only; no persistence. Use the - * Graph-backed implementation for persistence or indexing capabilities.

+ *

+ * Graph partitions share immutable RDF terms and statements. Transactions and lazy reads retain partition versions; + * writers detach shared index containers before mutation. Ownership changes and publication use a database monitor, + * while sequential snapshot traversal runs outside it. This does not make a single transaction or stream safe for + * concurrent use by multiple consumers. + * + *

+ * Callers must close streams returned by query operations, preferably with try-with-resources, even after a + * short-circuiting terminal operation. Completion of a terminal operation alone does not guarantee release of the + * captured state. Empty named graphs survive CLEAR and disappear on DROP. Batch ingestion reads its source once and + * publishes atomically after staging and index construction. + * + *

+ * This is the default in-memory backend selected by {@code Storages.create()} or through the {@code "memory"} + * storage configuration. */ package fr.inria.corese.core.next.storage.impl.memory; diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/plugin/MemoryStoragePlugin.java b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/plugin/MemoryStoragePlugin.java index ba0c98f65..5eb14193b 100644 --- a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/plugin/MemoryStoragePlugin.java +++ b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/plugin/MemoryStoragePlugin.java @@ -1,54 +1,62 @@ package fr.inria.corese.core.next.storage.impl.memory.plugin; import fr.inria.corese.core.next.storage.api.StorageManager; -import fr.inria.corese.core.next.storage.api.plugin.PluginException; -import fr.inria.corese.core.next.storage.api.plugin.StoragePlugin; import fr.inria.corese.core.next.storage.api.config.StorageConfig; +import fr.inria.corese.core.next.storage.api.plugin.StoragePlugin; import fr.inria.corese.core.next.storage.impl.memory.MemoryStorageManager; -import java.util.Objects; +/** Default native in-memory storage provider with SPO, POS and OSP indexes. */ +public final class MemoryStoragePlugin implements StoragePlugin { -/** - * Plugin for MemoryStorageManager - */ -public class MemoryStoragePlugin implements StoragePlugin { - - private static final String PLUGIN_NAME = "memory"; + /** Creates the stateless provider required by ServiceLoader. */ + public MemoryStoragePlugin() { + // The provider holds no state; create(StorageConfig) initializes each database. + } + /** + * {@inheritDoc} + * + * @return the plugin identifier memory + */ @Override public String getName() { - return PLUGIN_NAME; + return "memory"; } + /** + * {@inheritDoc} + * + * @return the backend description + */ @Override public String getDescription() { - return "In-memory backend for tests and small datasets (no persistence)"; + return "Native memory storage with SPO, POS and OSP indexes"; } + /** + * {@inheritDoc} + * + * @param config storage configuration selecting this backend + * @return true if the non-null configuration selects memory + */ @Override - public boolean supports(StorageConfig config) { - if (config == null) { - return false; - } - return config.getType() - .map(PLUGIN_NAME::equalsIgnoreCase) - .orElse(false); + public boolean supports(final StorageConfig config) { + return config != null && config.getType().map(getName()::equalsIgnoreCase).orElse(false); } + /** + * {@inheritDoc} + * + * @param config storage configuration selecting this backend + * @return an initialized owning native storage manager + */ @Override - public StorageManager create(StorageConfig config) throws PluginException { - StorageConfig checkedConfig = Objects.requireNonNull(config, "config"); - try { - MemoryStorageManager storage = MemoryStorageManager.builder().build(); - storage.lifecycle().initialize(checkedConfig); - return storage; - } catch (Exception e) { - throw new PluginException("Failed to create MemoryStorageManager", e); - } + public StorageManager create(final StorageConfig config) { + return new MemoryStorageManager(config); } @Override public int getPriority() { - return 50; + return 100; } } diff --git a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/plugin/package-info.java b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/plugin/package-info.java index 856a9dc4e..007bd889d 100644 --- a/src/main/java/fr/inria/corese/core/next/storage/impl/memory/plugin/package-info.java +++ b/src/main/java/fr/inria/corese/core/next/storage/impl/memory/plugin/package-info.java @@ -1,2 +1,5 @@ -/** Service provider for the built-in in-memory storage backend. */ +/** + * Default storage plugin provider for the native indexed in-memory backend. Providers are stateless; each created + * manager owns its independently initialized database. + */ package fr.inria.corese.core.next.storage.impl.memory.plugin; diff --git a/src/main/resources/META-INF/services/fr.inria.corese.core.next.storage.api.plugin.StoragePlugin b/src/main/resources/META-INF/services/fr.inria.corese.core.next.storage.api.plugin.StoragePlugin index f8461b5bc..1502d86a0 100644 --- a/src/main/resources/META-INF/services/fr.inria.corese.core.next.storage.api.plugin.StoragePlugin +++ b/src/main/resources/META-INF/services/fr.inria.corese.core.next.storage.api.plugin.StoragePlugin @@ -1,2 +1 @@ -fr.inria.corese.core.next.storage.impl.graph.plugin.GraphStoragePlugin -fr.inria.corese.core.next.storage.impl.memory.plugin.MemoryStoragePlugin \ No newline at end of file +fr.inria.corese.core.next.storage.impl.memory.plugin.MemoryStoragePlugin diff --git a/src/test/java/fr/inria/corese/core/next/data/api/term/SimpleIRITest.java b/src/test/java/fr/inria/corese/core/next/data/api/term/SimpleIRITest.java index 4c9f5ab38..73750d10f 100644 --- a/src/test/java/fr/inria/corese/core/next/data/api/term/SimpleIRITest.java +++ b/src/test/java/fr/inria/corese/core/next/data/api/term/SimpleIRITest.java @@ -56,6 +56,39 @@ void equalityAndHashCodeDependOnFullIRI() { assertNotEquals(FULL_IRI, full); } + @Test + void cachedHashPreservesCanonicalHashAndSurvivesSerialization() throws Exception { + // The supplied split need not be the canonical split inferred from the full IRI. + SimpleIRI iri = new SimpleIRI("http://example.org/", "path#term"); + var reference = new fr.inria.corese.core.next.data.spi.term.AbstractIRI(iri.stringValue()) { }; + assertEquals(reference.hashCode(), iri.hashCode()); + assertEquals(new SimpleIRI(iri.stringValue()).hashCode(), iri.hashCode()); + var bytes = new java.io.ByteArrayOutputStream(); + try (var output = new java.io.ObjectOutputStream(bytes)) { output.writeObject(iri); } + try (var input = new java.io.ObjectInputStream(new java.io.ByteArrayInputStream(bytes.toByteArray()))) { + SimpleIRI restored = (SimpleIRI) input.readObject(); + assertEquals(iri, restored); + assertEquals(reference.hashCode(), restored.hashCode()); + assertEquals(iri.hashCode(), restored.hashCode()); + } + } + + @Test + void readsIriSerializedBeforeHashCacheWasIntroduced() throws Exception { + // Generated by the original SimpleIRI class, with its unchanged serialVersionUID. + byte[] bytes = java.util.Base64.getDecoder().decode( + "rO0ABXNyADFmci5pbnJpYS5jb3Jlc2UuY29yZS5uZXh0LmRhdGEuYXBpLnRlcm0uU2ltcGxlSVJJ3hMxQG9xFt4C" + + "AAB4cgAzZnIuaW5yaWEuY29yZXNlLmNvcmUubmV4dC5kYXRhLnNwaS50ZXJtLkFic3RyYWN0SVJJ8gsYbESJkyEC" + + "AAJMAAlsb2NhbE5hbWV0ABJMamF2YS9sYW5nL1N0cmluZztMAAluYW1lc3BhY2VxAH4AAnhwdAAJcGF0aCN0ZXJt" + + "dAATaHR0cDovL2V4YW1wbGUub3JnLw=="); + try (var input = new java.io.ObjectInputStream(new java.io.ByteArrayInputStream(bytes))) { + SimpleIRI restored = (SimpleIRI) input.readObject(); + var reference = new fr.inria.corese.core.next.data.spi.term.AbstractIRI(restored.stringValue()) { }; + assertEquals("http://example.org/path#term", restored.stringValue()); + assertEquals(reference.hashCode(), restored.hashCode()); + } + } + @ParameterizedTest @NullAndEmptySource @ValueSource(strings = {" ", "relative/path", "http://example.org/invalid path"}) diff --git a/src/test/java/fr/inria/corese/core/next/data/impl/io/parser/support/ParserTestBase.java b/src/test/java/fr/inria/corese/core/next/data/impl/io/parser/support/ParserTestBase.java index f5d75a737..375db4c50 100644 --- a/src/test/java/fr/inria/corese/core/next/data/impl/io/parser/support/ParserTestBase.java +++ b/src/test/java/fr/inria/corese/core/next/data/impl/io/parser/support/ParserTestBase.java @@ -1,6 +1,5 @@ package fr.inria.corese.core.next.data.impl.io.parser.support; -import fr.inria.corese.core.Graph; import fr.inria.corese.core.next.data.Values; import fr.inria.corese.core.next.data.api.factory.ValueFactory; import fr.inria.corese.core.next.data.api.model.Model; @@ -40,21 +39,13 @@ protected void assertModelsIsomorphic(Model original, Model deserialized) { } /** - * Creates a test model using the production Graph backend. + * Creates a test model using the default native memory backend. * - * @return a new Model instance backed by Graph storage + * @return a new Model instance backed by memory storage */ protected Model createTestModel() { - Graph graph = Graph.create(); - - StorageConfig config = StorageConfig.builder() - .property("type", "graph") - .property("graph", graph) - .property("valueFactory", valueFactory) - .build(); - return StorageModel.builder() - .storage(Storages.create(config)) + .storage(Storages.create()) .valueFactory(valueFactory) .build(); } diff --git a/src/test/java/fr/inria/corese/core/next/data/impl/model/FilteredModelTest.java b/src/test/java/fr/inria/corese/core/next/data/impl/model/FilteredModelTest.java index caebb5254..521ec0d60 100644 --- a/src/test/java/fr/inria/corese/core/next/data/impl/model/FilteredModelTest.java +++ b/src/test/java/fr/inria/corese/core/next/data/impl/model/FilteredModelTest.java @@ -62,8 +62,6 @@ void testConstructorNullChecks() { @Test void testAddMatchingStatement() { - baseModel.add(subjectFilter, predicateFilter, objectFilter); - assertTrue(filteredModel.add(subjectFilter, predicateFilter, objectFilter)); assertTrue(baseModel.contains(subjectFilter, predicateFilter, objectFilter)); } diff --git a/src/test/java/fr/inria/corese/core/next/query/impl/repository/CoreseUpdateExecutionTest.java b/src/test/java/fr/inria/corese/core/next/query/impl/repository/CoreseUpdateExecutionTest.java index 61571d248..bd0a61065 100644 --- a/src/test/java/fr/inria/corese/core/next/query/impl/repository/CoreseUpdateExecutionTest.java +++ b/src/test/java/fr/inria/corese/core/next/query/impl/repository/CoreseUpdateExecutionTest.java @@ -1,6 +1,5 @@ package fr.inria.corese.core.next.query.impl.repository; -import fr.inria.corese.core.Graph; import fr.inria.corese.core.next.data.Values; import fr.inria.corese.core.next.data.api.term.IRI; import fr.inria.corese.core.next.data.api.term.Resource; @@ -10,7 +9,6 @@ import fr.inria.corese.core.next.query.api.repository.Repository; import fr.inria.corese.core.next.query.api.repository.RepositoryConnection; import fr.inria.corese.core.next.storage.api.StorageManager; -import fr.inria.corese.core.next.storage.impl.graph.GraphStorageManager; import fr.inria.corese.core.next.storage.impl.memory.MemoryStorageManager; import org.junit.jupiter.api.AfterEach; import org.junit.jupiter.api.BeforeEach; @@ -238,21 +236,7 @@ void illegalDataAndDeleteTemplatesAreRejectedBeforeExecution() { assertEquals(0, connection.size()); } - @Test - void legacyGraphBackendSupportsUpdatesAndFailureRecovery() { - StorageManager legacy = GraphStorageManager.builder().graph(Graph.create()).valueFactory(Values.factory()).build(); - try (Repository repo = Repositories.create(legacy); var conn = repo.getConnection()) { - conn.prepareUpdate(PREFIX + "CREATE GRAPH ex:g; INSERT DATA { GRAPH ex:g { ex:s ex:p ex:o } }").execute(); - assertEquals(1, conn.size()); - var failingUpdate = conn.prepareUpdate(PREFIX + - "CLEAR ALL; DROP GRAPH ex:g; COPY GRAPH ex:missing TO DEFAULT"); - assertThrows(QueryEvaluationException.class, failingUpdate::execute); - assertEquals(1, conn.size()); - assertTrue(legacy.metadata().getContexts().contains(iri("g"))); - conn.prepareUpdate(PREFIX + "DROP ALL").execute(); - assertEquals(0, conn.size()); - } - } + @Test void eachOperationRetainsItsPrefixAndBaseScope() { diff --git a/src/test/java/fr/inria/corese/core/next/query/impl/repository/PreparedQueryReuseTest.java b/src/test/java/fr/inria/corese/core/next/query/impl/repository/PreparedQueryReuseTest.java new file mode 100644 index 000000000..f1a35ec46 --- /dev/null +++ b/src/test/java/fr/inria/corese/core/next/query/impl/repository/PreparedQueryReuseTest.java @@ -0,0 +1,227 @@ +package fr.inria.corese.core.next.query.impl.repository; + +import fr.inria.corese.core.next.data.Values; +import fr.inria.corese.core.next.data.api.model.Statement; +import fr.inria.corese.core.next.data.api.term.BNode; +import fr.inria.corese.core.next.data.api.term.IRI; +import fr.inria.corese.core.next.query.Repositories; +import fr.inria.corese.core.next.query.api.GraphQuery; +import fr.inria.corese.core.next.query.api.TupleQuery; +import fr.inria.corese.core.next.query.api.dataset.Dataset; +import fr.inria.corese.core.next.query.api.exception.RepositoryException; +import fr.inria.corese.core.next.query.api.exception.QueryTimeoutException; +import fr.inria.corese.core.next.query.api.exception.QueryEvaluationException; +import fr.inria.corese.core.next.storage.Storages; +import fr.inria.corese.core.next.storage.api.config.StorageConfig; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.ValueSource; + +import java.util.ArrayList; +import java.util.List; +import java.time.Duration; + +import static org.junit.jupiter.api.Assertions.*; + +class PreparedQueryReuseTest { + private static IRI iri(String local) { + return Values.factory().createIRI("https://example.org/" + local); + } + + private static List objects(TupleQuery query) { + List values = new ArrayList<>(); + try (var rows = query.evaluate()) { + while (rows.hasNext()) values.add(rows.next().getValue("o").stringValue()); + } + values.sort(String::compareTo); + return values; + } + + private static List statements(GraphQuery query) { + try (var rows = query.evaluate()) { + return rows.stream().toList(); + } + } + + @ParameterizedTest + @ValueSource(strings = {"memory"}) + void repeatedSelectUsesCurrentBindingsDatasetAndData(String backend) { + try (var repository = Repositories.create(Storages.create(StorageConfig.builder().type(backend).build())); + var connection = repository.getConnection()) { + connection.add(iri("alice"), iri("p"), iri("bob"), iri("g1")); + connection.add(iri("carol"), iri("p"), iri("dave"), iri("g2")); + var query = connection.prepareTupleQuery( + "SELECT ?o FROM { ?s

?o }", "https://example.org/query.rq"); + assertEquals(List.of(iri("bob").stringValue()), objects(query)); + query.setDataset(Dataset.builder().defaultGraph(iri("g2")).build()); + assertEquals(List.of(iri("dave").stringValue()), objects(query)); + query.setBinding("s", iri("alice")); + assertTrue(objects(query).isEmpty()); + query.setBinding("s", iri("carol")); + assertEquals(List.of(iri("dave").stringValue()), objects(query)); + connection.add(iri("carol"), iri("p"), iri("eve"), iri("g2")); + assertEquals(List.of(iri("dave").stringValue(), iri("eve").stringValue()), objects(query)); + query.clearBindings(); + query.setDataset(null); + assertEquals(List.of(iri("bob").stringValue()), objects(query)); + connection.close(); + assertThrows(RepositoryException.class, query::evaluate); + } + } + + @ParameterizedTest + @ValueSource(strings = {"memory"}) + void repeatedAskUsesCurrentBindingsDatasetAndData(String backend) { + try (var repository = Repositories.create(Storages.create(StorageConfig.builder().type(backend).build())); + var connection = repository.getConnection()) { + connection.add(iri("alice"), iri("p"), iri("bob"), iri("g1")); + var query = connection.prepareBooleanQuery( + "ASK FROM { ?s

?o }", "https://example.org/query.rq"); + assertTrue(query.evaluate()); + query.setBinding("o", iri("eve")); + assertFalse(query.evaluate()); + connection.add(iri("alice"), iri("p"), iri("eve"), iri("g1")); + assertTrue(query.evaluate()); + query.setDataset(Dataset.empty()); + assertFalse(query.evaluate()); + query.clearBindings(); + query.setDataset(null); + assertTrue(query.evaluate()); + } + } + + @ParameterizedTest + @ValueSource(strings = {"memory"}) + void repeatedConstructCreatesFreshBlankNodesAndUsesCurrentOptions(String backend) { + try (var repository = Repositories.create(Storages.create(StorageConfig.builder().type(backend).build())); + var connection = repository.getConnection()) { + connection.add(iri("alice"), iri("p"), iri("bob"), iri("g1")); + connection.add(iri("alice"), iri("p"), iri("eve"), iri("g2")); + var query = connection.prepareGraphQuery( + "CONSTRUCT { _:b

?o } FROM WHERE { ?s

?o }", + "https://example.org/query.rq"); + var first = statements(query); + var second = statements(query); + assertEquals(1, first.size()); + assertEquals(1, second.size()); + assertEquals(iri("bob"), first.getFirst().getObject()); + assertInstanceOf(BNode.class, first.getFirst().getSubject()); + assertNotEquals(first.getFirst().getSubject(), second.getFirst().getSubject()); + query.setDataset(Dataset.builder().defaultGraph(iri("g2")).build()); + assertEquals(iri("eve"), statements(query).getFirst().getObject()); + query.setBinding("o", iri("bob")); + assertTrue(statements(query).isEmpty()); + query.clearBindings(); + query.setDataset(null); + assertEquals(iri("bob"), statements(query).getFirst().getObject()); + } + } + + @ParameterizedTest + @ValueSource(strings = {"memory"}) + void repeatedDescribeSeesMutationsAndDatasetOverrides(String backend) { + try (var repository = Repositories.create(Storages.create(StorageConfig.builder().type(backend).build())); + var connection = repository.getConnection()) { + connection.add(iri("alice"), iri("p"), iri("bob"), iri("g1")); + var query = connection.prepareGraphQuery( + "DESCRIBE FROM ", "https://example.org/query.rq"); + assertEquals(1, statements(query).size()); + connection.add(iri("alice"), iri("p"), iri("eve"), iri("g1")); + assertEquals(2, statements(query).size()); + query.setDataset(Dataset.empty()); + assertTrue(statements(query).isEmpty()); + query.setDataset(null); + assertEquals(2, statements(query).size()); + } + } + + @ParameterizedTest + @ValueSource(strings = {"memory"}) + void evaluationAfterTimeoutUsesTheNewDeadline(String backend) { + try (var repository = Repositories.create(Storages.create(StorageConfig.builder().type(backend).build())); + var connection = repository.getConnection()) { + for (int i = 0; i < 201; i++) connection.add(iri("s" + i), iri("p"), iri("o" + i)); + var query = connection.prepareTupleQuery("SELECT * { ?s1 ?p1 ?o1 . ?s2 ?p2 ?o2 }"); + query.setTimeout(Duration.ofMillis(1)); + assertThrows(QueryTimeoutException.class, () -> { + try (var rows = query.evaluate()) { rows.stream().count(); } + }); + query.setTimeout(Duration.ofSeconds(30)); + try (var rows = query.evaluate()) { assertEquals(40_401, rows.stream().count()); } + } + } + + @ParameterizedTest + @ValueSource(strings = {"memory"}) + void repeatedEvaluationRecomputesVolatileExpressions(String backend) { + try (var repository = Repositories.create(Storages.create(StorageConfig.builder().type(backend).build())); + var connection = repository.getConnection()) { + var query = connection.prepareTupleQuery("SELECT (UUID() AS ?o) WHERE {}"); + assertNotEquals(objects(query), objects(query)); + } + } + + @ParameterizedTest + @ValueSource(strings = {"memory"}) + void evaluationWithTimeoutInsideActiveTransactionSucceeds(String backend) { + try (var repository = Repositories.create(Storages.create(StorageConfig.builder().type(backend).build())); + var connection = repository.getConnection()) { + connection.begin(); + connection.add(iri("s"), iri("p"), iri("o")); + var ask = connection.prepareBooleanQuery("ASK { ?s ?p ?o }"); + ask.setTimeout(Duration.ofSeconds(10)); + assertTrue(ask.evaluate()); + + var tuple = connection.prepareTupleQuery("SELECT ?o { ?s ?p ?o }"); + tuple.setTimeout(Duration.ofSeconds(10)); + assertEquals(List.of(iri("o").stringValue()), objects(tuple)); + + connection.rollback(); + } + } + + @ParameterizedTest + @ValueSource(strings = {"memory"}) + void expiredQueryLeavesTransactionUsableAndCallerUninterrupted(String backend) { + try (var repository = Repositories.create(Storages.create(StorageConfig.builder().type(backend).build())); + var connection = repository.getConnection()) { + connection.begin(); + for (int i = 0; i < 201; i++) { + connection.add(iri("s" + i), iri("p"), iri("o" + i)); + } + var query = connection.prepareTupleQuery("SELECT * { ?s1 ?p1 ?o1 . ?s2 ?p2 ?o2 }"); + query.setTimeout(Duration.ofMillis(1)); + assertThrows(QueryTimeoutException.class, () -> { + try (var rows = query.evaluate()) { + rows.stream().count(); + } + }); + assertFalse(Thread.currentThread().isInterrupted()); + assertTrue(connection.isActive()); + + var ask = connection.prepareBooleanQuery("ASK { ?s ?p ?o }"); + ask.setTimeout(Duration.ofSeconds(10)); + assertTrue(ask.evaluate()); + connection.commit(); + assertEquals(201, connection.size()); + } + } + + @ParameterizedTest + @ValueSource(strings = {"memory"}) + void callerInterruptionIsPreservedWhenEvaluationFails(String backend) { + try (var repository = Repositories.create(Storages.create(StorageConfig.builder().type(backend).build())); + var connection = repository.getConnection()) { + connection.add(iri("s"), iri("p"), iri("o")); + var ask = connection.prepareBooleanQuery("ASK { ?s ?p ?o }"); + ask.setTimeout(Duration.ofSeconds(10)); + try { + Thread.currentThread().interrupt(); + assertThrows(QueryEvaluationException.class, ask::evaluate); + assertTrue(Thread.currentThread().isInterrupted()); + } finally { + Thread.interrupted(); + } + assertTrue(ask.evaluate()); + } + } +} diff --git a/src/test/java/fr/inria/corese/core/next/query/impl/sparql/parser/SparqlParserTripleOrderTest.java b/src/test/java/fr/inria/corese/core/next/query/impl/sparql/parser/SparqlParserTripleOrderTest.java new file mode 100644 index 000000000..c5e51d468 --- /dev/null +++ b/src/test/java/fr/inria/corese/core/next/query/impl/sparql/parser/SparqlParserTripleOrderTest.java @@ -0,0 +1,28 @@ +package fr.inria.corese.core.next.query.impl.sparql.parser; + +import fr.inria.corese.core.next.query.impl.sparql.ast.BgpAst; +import fr.inria.corese.core.next.query.impl.sparql.ast.SelectQueryAst; +import fr.inria.corese.core.next.query.impl.sparql.ast.VarAst; +import org.junit.jupiter.api.Test; + +import java.util.List; + +import static org.junit.jupiter.api.Assertions.assertEquals; + +class SparqlParserTripleOrderTest { + @Test + void recursiveTriplesBlocksPreserveSourceOrder() { + var query = (SelectQueryAst) new SparqlParser().parse(""" + PREFIX : + SELECT ?s ?friend ?name WHERE { + ?s :group :g7 ; :knows ?friend . + ?friend :knows ?other . + ?other :name ?name . + } + """); + List subjects = query.whereClause().patterns().stream() + .map(BgpAst.class::cast) + .map(bgp -> ((VarAst) bgp.triples().getFirst().subject()).name()).toList(); + assertEquals(List.of("s", "friend", "other"), subjects); + } +} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/graph/CoreseGraphStatementStoreTest.java b/src/test/java/fr/inria/corese/core/next/storage/impl/graph/CoreseGraphStatementStoreTest.java deleted file mode 100644 index 8c822d75a..000000000 --- a/src/test/java/fr/inria/corese/core/next/storage/impl/graph/CoreseGraphStatementStoreTest.java +++ /dev/null @@ -1,156 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.graph; - -import fr.inria.corese.core.Graph; -import fr.inria.corese.core.next.data.api.model.Statement; -import fr.inria.corese.core.next.data.api.factory.ValueFactory; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.DisplayName; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; -import org.mockito.Mock; -import org.mockito.MockitoAnnotations; - -import static org.junit.jupiter.api.Assertions.*; -import static org.mockito.Mockito.verify; -import static org.mockito.Mockito.when; - -/** - * Unit tests for CoreseGraphStatementStore. - */ -class CoreseGraphStatementStoreTest { - - @Mock - private Graph mockGraph; - - @Mock - private ValueFactory mockFactory; - - private CoreseGraphStatementStore adapter; - - @BeforeEach - void setUp() { - MockitoAnnotations.openMocks(this); - adapter = new CoreseGraphStatementStore(mockGraph, mockFactory); - } - - @Nested - @DisplayName("Constructor tests") - class ConstructorTests { - - @Test - @DisplayName("Should throw when graph is null") - void shouldThrowWhenGraphIsNull() { - assertThrows(IllegalArgumentException.class, - () -> new CoreseGraphStatementStore(null, mockFactory)); - } - - @Test - @DisplayName("Should throw when valueFactory is null") - void shouldThrowWhenValueFactoryIsNull() { - assertThrows(IllegalArgumentException.class, - () -> new CoreseGraphStatementStore(mockGraph, null)); - } - - @Test - @DisplayName("Should create adapter with valid parameters") - void shouldCreateAdapterWithValidParameters() { - CoreseGraphStatementStore createdStore = new CoreseGraphStatementStore(mockGraph, mockFactory); - assertNotNull(createdStore); - assertEquals(mockGraph, createdStore.graph()); - assertEquals(mockFactory, createdStore.valueFactory()); - } - } - - @Nested - @DisplayName("CRUD operations") - class CrudOperationsTests { - - @Mock - private Statement mockStatement; - - @Test - @DisplayName("Should add statement successfully") - void shouldAddStatementSuccessfully() { - assertTrue(true, "Test placeholder - requires Graph API mocking"); - } - - @Test - @DisplayName("Should return size from graph") - void shouldReturnSizeFromGraph() { - when(mockGraph.size()).thenReturn(42); - - int size = adapter.size(); - - assertEquals(42, size); - verify(mockGraph).size(); - } - - @Test - @DisplayName("Should clear graph") - void shouldClearGraph() { - adapter.clear(); - - verify(mockGraph).clear(); - } - } - - @Nested - @DisplayName("Metadata operations") - class MetadataOperationsTests { - - @Test - @DisplayName("Should get subjects from graph") - void shouldGetSubjectsFromGraph() { - when(mockGraph.getEdges()).thenReturn(java.util.Collections.emptyList()); - - var subjects = adapter.getSubjects(); - - assertNotNull(subjects); - verify(mockGraph).getEdges(); - } - - @Test - @DisplayName("Should get predicates from graph") - void shouldGetPredicatesFromGraph() { - when(mockGraph.getEdges()).thenReturn(java.util.Collections.emptyList()); - - var predicates = adapter.getPredicates(); - - assertNotNull(predicates); - verify(mockGraph).getEdges(); - } - - @Test - @DisplayName("Should get objects from graph") - void shouldGetObjectsFromGraph() { - when(mockGraph.getEdges()).thenReturn(java.util.Collections.emptyList()); - - var objects = adapter.getObjects(); - - assertNotNull(objects); - verify(mockGraph).getEdges(); - } - - @Test - @DisplayName("Should get contexts from graph") - void shouldGetContextsFromGraph() { - when(mockGraph.getGraphNodes()).thenReturn(java.util.Collections.emptyList()); - - var contexts = adapter.getContexts(); - - assertNotNull(contexts); - verify(mockGraph).getGraphNodes(); - } - } - - @Nested - @DisplayName("Conversion methods") - class ConversionMethodsTests { - - @Test - @DisplayName("Conversion methods exist and are private") - void conversionMethodsExistAndArePrivate() { - assertTrue(true, "Conversion methods tested through integration tests"); - } - } -} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/graph/GraphLifecycleManagerTest.java b/src/test/java/fr/inria/corese/core/next/storage/impl/graph/GraphLifecycleManagerTest.java deleted file mode 100644 index 10ae053a0..000000000 --- a/src/test/java/fr/inria/corese/core/next/storage/impl/graph/GraphLifecycleManagerTest.java +++ /dev/null @@ -1,120 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.graph; - -import fr.inria.corese.core.Graph; -import fr.inria.corese.core.next.storage.api.lifecycle.LifecycleState; -import fr.inria.corese.core.next.storage.api.config.StorageConfig; -import fr.inria.corese.core.next.storage.api.exception.StorageException; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.DisplayName; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; -import org.mockito.Mock; -import org.mockito.MockitoAnnotations; - -import static org.junit.jupiter.api.Assertions.*; -import static org.mockito.Mockito.doThrow; -import static org.mockito.Mockito.verify; - -@DisplayName("GraphLifecycleManager tests") -class GraphLifecycleManagerTest { - - @Mock - private Graph mockGraph; - - private GraphLifecycleManager lifecycleManager; - - @BeforeEach - void setUp() { - MockitoAnnotations.openMocks(this); - lifecycleManager = new GraphLifecycleManager(mockGraph); - } - - @Nested - @DisplayName("Constructor tests") - class ConstructorTests { - - @Test - @DisplayName("Should throw when graph is null") - void shouldThrowWhenGraphIsNull() { - assertThrows(IllegalArgumentException.class, - () -> new GraphLifecycleManager(null)); - } - - @Test - @DisplayName("Should start in NOT_INITIALIZED state") - void shouldStartInNotInitializedState() { - assertEquals(LifecycleState.NOT_INITIALIZED, - lifecycleManager.getState()); - } - } - - @Nested - @DisplayName("initialize() tests") - class InitializeTests { - - @Test - @DisplayName("Should throw when config is null") - void shouldThrowWhenConfigIsNull() { - assertThrows(IllegalArgumentException.class, - () -> lifecycleManager.initialize(null)); - } - - @Test - @DisplayName("Should initialize successfully") - void shouldInitializeSuccessfully() throws StorageException { - StorageConfig config = StorageConfig.builder().build(); - - lifecycleManager.initialize(config); - - assertEquals(LifecycleState.RUNNING, lifecycleManager.getState()); - assertEquals(java.util.Optional.of(config), lifecycleManager.getConfig()); - verify(mockGraph).init(); - } - - @Test - @DisplayName("Should throw when already initialized") - void shouldThrowWhenAlreadyInitialized() throws StorageException { - StorageConfig config = StorageConfig.builder().build(); - lifecycleManager.initialize(config); - - assertThrows(IllegalStateException.class, - () -> lifecycleManager.initialize(config)); - } - - @Test - @DisplayName("Should remain NOT_INITIALIZED on error") - void shouldRemainNotInitializedOnError() { - StorageConfig config = StorageConfig.builder().build(); - doThrow(new RuntimeException("Init failed")).when(mockGraph).init(); - - assertThrows(StorageException.class, - () -> lifecycleManager.initialize(config)); - assertEquals(LifecycleState.NOT_INITIALIZED, - lifecycleManager.getState()); - } - } - - @Nested - @DisplayName("shutdown() tests") - class ShutdownTests { - - @Test - @DisplayName("Should throw when not running") - void shouldThrowWhenNotRunning() { - assertThrows(IllegalStateException.class, - () -> lifecycleManager.shutdown()); - } - - @Test - @DisplayName("Should shutdown successfully") - void shouldShutdownSuccessfully() throws StorageException { - StorageConfig config = StorageConfig.builder().build(); - lifecycleManager.initialize(config); - - lifecycleManager.shutdown(); - - assertEquals(LifecycleState.SHUTDOWN, lifecycleManager.getState()); - assertTrue(lifecycleManager.getConfig().isEmpty()); - } - } -} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/graph/GraphMetadataOperationsTest.java b/src/test/java/fr/inria/corese/core/next/storage/impl/graph/GraphMetadataOperationsTest.java deleted file mode 100644 index 675740ef4..000000000 --- a/src/test/java/fr/inria/corese/core/next/storage/impl/graph/GraphMetadataOperationsTest.java +++ /dev/null @@ -1,282 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.graph; - -import fr.inria.corese.core.next.data.api.term.*; -import fr.inria.corese.core.next.storage.api.exception.StorageException; -import fr.inria.corese.core.next.storage.api.model.StorageStatistics; -import org.junit.jupiter.api.*; -import org.mockito.Mock; -import org.mockito.MockitoAnnotations; - -import java.util.*; - -import static org.junit.jupiter.api.Assertions.*; -import static org.mockito.Mockito.*; - -/** - * Unit tests for GraphMetadataOperations. - */ -class GraphMetadataOperationsTest { - - @Mock - private CoreseGraphStatementStore mockAdapter; - - private GraphMetadataOperations metadataOps; - - @BeforeEach - void setUp() { - MockitoAnnotations.openMocks(this); - metadataOps = new GraphMetadataOperations(mockAdapter); - } - - @Nested - @DisplayName("Constructor tests") - class ConstructorTests { - - @Test - @DisplayName("Should throw when adapter is null") - void shouldThrowWhenAdapterIsNull() { - assertThrows(IllegalArgumentException.class, - () -> new GraphMetadataOperations(null)); - } - } - - @Nested - @DisplayName("getPredicates() tests") - class GetPredicatesTests { - - @Test - @DisplayName("Should return unmodifiable set of predicates") - void shouldReturnUnmodifiableSetOfPredicates() throws StorageException { - IRI predicate1 = mock(IRI.class); - IRI predicate2 = mock(IRI.class); - Set mockPredicates = new HashSet<>(Arrays.asList(predicate1, predicate2)); - - when(mockAdapter.getPredicates()).thenReturn(mockPredicates); - - Set result = metadataOps.getPredicates(); - - assertEquals(2, result.size()); - assertTrue(result.contains(predicate1)); - assertTrue(result.contains(predicate2)); - verify(mockAdapter).getPredicates(); - } - - @Test - @DisplayName("Should return unmodifiable set") - void shouldReturnUnmodifiableSet() throws StorageException { - when(mockAdapter.getPredicates()).thenReturn(new HashSet<>()); - - Set result = metadataOps.getPredicates(); - - assertThrows(UnsupportedOperationException.class, - () -> result.add(mock(IRI.class))); - } - - @Test - @DisplayName("Should return empty set when no predicates") - void shouldReturnEmptySetWhenNoPredicates() throws StorageException { - when(mockAdapter.getPredicates()).thenReturn(Collections.emptySet()); - - Set result = metadataOps.getPredicates(); - - assertTrue(result.isEmpty()); - } - } - - @Nested - @DisplayName("getSubjects() tests") - class GetSubjectsTests { - - @Test - @DisplayName("Should return unmodifiable set of subjects") - void shouldReturnUnmodifiableSetOfSubjects() throws StorageException { - Resource subject1 = mock(Resource.class); - Resource subject2 = mock(Resource.class); - Set mockSubjects = new HashSet<>(Arrays.asList(subject1, subject2)); - - when(mockAdapter.getSubjects()).thenReturn(mockSubjects); - - Set result = metadataOps.getSubjects(); - - assertEquals(2, result.size()); - assertTrue(result.contains(subject1)); - assertTrue(result.contains(subject2)); - verify(mockAdapter).getSubjects(); - } - - @Test - @DisplayName("Should return unmodifiable set") - void shouldReturnUnmodifiableSet() throws StorageException { - when(mockAdapter.getSubjects()).thenReturn(new HashSet<>()); - - Set result = metadataOps.getSubjects(); - - assertThrows(UnsupportedOperationException.class, - () -> result.add(mock(Resource.class))); - } - } - - @Nested - @DisplayName("getObjects() tests") - class GetObjectsTests { - - @Test - @DisplayName("Should return unmodifiable set of objects") - void shouldReturnUnmodifiableSetOfObjects() throws StorageException { - Value object1 = mock(Value.class); - Value object2 = mock(Value.class); - Value object3 = mock(Value.class); - Set mockObjects = new HashSet<>(Arrays.asList(object1, object2, object3)); - - when(mockAdapter.getObjects()).thenReturn(mockObjects); - - Set result = metadataOps.getObjects(); - - assertEquals(3, result.size()); - assertTrue(result.contains(object1)); - assertTrue(result.contains(object2)); - assertTrue(result.contains(object3)); - verify(mockAdapter).getObjects(); - } - - @Test - @DisplayName("Should return unmodifiable set") - void shouldReturnUnmodifiableSet() throws StorageException { - when(mockAdapter.getObjects()).thenReturn(new HashSet<>()); - - Set result = metadataOps.getObjects(); - - assertThrows(UnsupportedOperationException.class, - () -> result.add(mock(Value.class))); - } - } - - @Nested - @DisplayName("getContexts() tests") - class GetContextsTests { - - @Test - @DisplayName("Should return unmodifiable set of contexts") - void shouldReturnUnmodifiableSetOfContexts() throws StorageException { - Resource context1 = mock(Resource.class); - Resource context2 = mock(Resource.class); - Set mockContexts = new HashSet<>(Arrays.asList(context1, context2)); - - when(mockAdapter.getContexts()).thenReturn(mockContexts); - - Set result = metadataOps.getContexts(); - - assertEquals(2, result.size()); - assertTrue(result.contains(context1)); - assertTrue(result.contains(context2)); - verify(mockAdapter).getContexts(); - } - - @Test - @DisplayName("Should return unmodifiable set") - void shouldReturnUnmodifiableSet() throws StorageException { - when(mockAdapter.getContexts()).thenReturn(new HashSet<>()); - - Set result = metadataOps.getContexts(); - - assertThrows(UnsupportedOperationException.class, - () -> result.add(mock(Resource.class))); - } - - @Test - @DisplayName("Should return empty set when no contexts") - void shouldReturnEmptySetWhenNoContexts() throws StorageException { - when(mockAdapter.getContexts()).thenReturn(Collections.emptySet()); - - Set result = metadataOps.getContexts(); - - assertTrue(result.isEmpty()); - } - } - - @Nested - @DisplayName("getStatistics() tests") - class GetStatisticsTests { - - @Test - @DisplayName("Should return statistics with correct counts") - void shouldReturnStatisticsWithCorrectCounts() throws StorageException { - // Setup mocks - when(mockAdapter.size()).thenReturn(100); - - Set subjects = new HashSet<>(Arrays.asList( - mock(Resource.class), mock(Resource.class), mock(Resource.class) - )); - when(mockAdapter.getSubjects()).thenReturn(subjects); - - Set predicates = new HashSet<>(Arrays.asList( - mock(IRI.class), mock(IRI.class) - )); - when(mockAdapter.getPredicates()).thenReturn(predicates); - - Set objects = new HashSet<>(Arrays.asList( - mock(Value.class), mock(Value.class), mock(Value.class), mock(Value.class) - )); - when(mockAdapter.getObjects()).thenReturn(objects); - - Set contexts = new HashSet<>(Collections.singletonList( - mock(Resource.class) - )); - when(mockAdapter.getContexts()).thenReturn(contexts); - - // Get statistics - StorageStatistics stats = metadataOps.getStatistics(); - - // Verify - assertEquals(100, stats.statementCount()); - assertEquals(3, stats.subjectCount()); - assertEquals(2, stats.predicateCount()); - assertEquals(4, stats.objectCount()); - assertEquals(1, stats.contextCount()); - - verify(mockAdapter).size(); - verify(mockAdapter).getSubjects(); - verify(mockAdapter).getPredicates(); - verify(mockAdapter).getObjects(); - verify(mockAdapter).getContexts(); - } - - @Test - @DisplayName("Should return zero statistics for empty graph") - void shouldReturnZeroStatisticsForEmptyGraph() throws StorageException { - when(mockAdapter.size()).thenReturn(0); - when(mockAdapter.getSubjects()).thenReturn(Collections.emptySet()); - when(mockAdapter.getPredicates()).thenReturn(Collections.emptySet()); - when(mockAdapter.getObjects()).thenReturn(Collections.emptySet()); - when(mockAdapter.getContexts()).thenReturn(Collections.emptySet()); - - StorageStatistics stats = metadataOps.getStatistics(); - - assertEquals(0, stats.statementCount()); - assertEquals(0, stats.subjectCount()); - assertEquals(0, stats.predicateCount()); - assertEquals(0, stats.objectCount()); - assertEquals(0, stats.contextCount()); - } - - @Test - @DisplayName("Should call adapter methods for statistics") - void shouldCallAdapterMethodsForStatistics() throws StorageException { - // Setup with empty sets to simplify - when(mockAdapter.size()).thenReturn(0); - when(mockAdapter.getSubjects()).thenReturn(Collections.emptySet()); - when(mockAdapter.getPredicates()).thenReturn(Collections.emptySet()); - when(mockAdapter.getObjects()).thenReturn(Collections.emptySet()); - when(mockAdapter.getContexts()).thenReturn(Collections.emptySet()); - - metadataOps.getStatistics(); - - // Verify all methods were called - verify(mockAdapter, times(1)).size(); - verify(mockAdapter, times(1)).getSubjects(); - verify(mockAdapter, times(1)).getPredicates(); - verify(mockAdapter, times(1)).getObjects(); - verify(mockAdapter, times(1)).getContexts(); - } - } -} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/graph/GraphMutationOperationsTest.java b/src/test/java/fr/inria/corese/core/next/storage/impl/graph/GraphMutationOperationsTest.java deleted file mode 100644 index 3c2af72f2..000000000 --- a/src/test/java/fr/inria/corese/core/next/storage/impl/graph/GraphMutationOperationsTest.java +++ /dev/null @@ -1,83 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.graph; - -import fr.inria.corese.core.next.data.api.model.Statement; -import fr.inria.corese.core.next.data.api.term.Resource; -import fr.inria.corese.core.next.storage.api.exception.StorageException; -import fr.inria.corese.core.next.storage.api.model.StatementPattern; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.Test; - -import java.util.Set; - -import static org.junit.jupiter.api.Assertions.*; -import static org.mockito.ArgumentMatchers.*; -import static org.mockito.Mockito.*; - -class GraphMutationOperationsTest { - - private CoreseGraphStatementStore store; - private GraphMutationOperations mutations; - - @BeforeEach - void setUp() { - store = mock(CoreseGraphStatementStore.class); - mutations = new GraphMutationOperations(store); - } - - @Test - void rejectsMissingDependenciesAndArguments() { - assertThrows(IllegalArgumentException.class, () -> new GraphMutationOperations(null)); - assertThrows(NullPointerException.class, () -> mutations.add(null)); - assertThrows(NullPointerException.class, () -> mutations.remove((Statement) null)); - assertThrows(NullPointerException.class, () -> mutations.remove((StatementPattern) null)); - } - - @Test - void reportsWhetherSingleStatementMutationsChangedTheStore() { - Statement statement = mock(Statement.class); - when(store.add(statement)).thenReturn(true, false); - when(store.remove(statement)).thenReturn(true, false); - - assertTrue(mutations.add(statement)); - assertFalse(mutations.add(statement)); - assertTrue(mutations.remove(statement)); - assertFalse(mutations.remove(statement)); - } - - @Test - void wrapsBackendFailures() { - Statement statement = mock(Statement.class); - when(store.add(statement)).thenThrow(new IllegalStateException("failed")); - - assertThrows(StorageException.class, () -> mutations.add(statement)); - } - - @Test - void removesMatchingStatementsAndReturnsTheirCount() { - Statement first = mock(Statement.class); - Statement second = mock(Statement.class); - when(store.find(isNull(), isNull(), isNull(), any(Resource[].class))) - .thenReturn(Set.of(first, second)); - - assertEquals(2, mutations.remove(StatementPattern.matchAll())); - verify(store).remove(first); - verify(store).remove(second); - } - - @Test - void clearsAllOrSelectedGraphsAndReturnsTheRemovedCount() { - Resource first = mock(Resource.class); - Resource second = mock(Resource.class); - Statement firstMatch = mock(Statement.class); - Statement secondMatch = mock(Statement.class); - when(store.size()).thenReturn(10, 0); - when(store.find(null, null, null, new Resource[]{first, second})) - .thenReturn(Set.of(firstMatch, secondMatch)); - - assertEquals(10, mutations.clear()); - assertEquals(2, mutations.clear(first, second)); - verify(store).clear(); - verify(store).remove(firstMatch); - verify(store).remove(secondMatch); - } -} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/graph/GraphQueryOperationsTest.java b/src/test/java/fr/inria/corese/core/next/storage/impl/graph/GraphQueryOperationsTest.java deleted file mode 100644 index 661745d0b..000000000 --- a/src/test/java/fr/inria/corese/core/next/storage/impl/graph/GraphQueryOperationsTest.java +++ /dev/null @@ -1,148 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.graph; - -import fr.inria.corese.core.next.data.api.model.Statement; -import fr.inria.corese.core.next.storage.api.exception.StorageException; -import fr.inria.corese.core.next.storage.api.model.StatementPattern; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.DisplayName; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; -import org.mockito.Mock; -import org.mockito.MockitoAnnotations; - -import java.util.Arrays; -import java.util.Collections; -import java.util.HashSet; -import java.util.Set; -import java.util.stream.Stream; - -import static org.junit.jupiter.api.Assertions.*; -import static org.mockito.Mockito.*; - -/** - * Unit tests for GraphQueryOperations. - */ -class GraphQueryOperationsTest { - - @Mock - private CoreseGraphStatementStore mockAdapter; - - private GraphQueryOperations queryOps; - - @BeforeEach - void setUp() { - MockitoAnnotations.openMocks(this); - queryOps = new GraphQueryOperations(mockAdapter); - } - - @Nested - @DisplayName("Constructor tests") - class ConstructorTests { - - @Test - @DisplayName("Should throw when adapter is null") - void shouldThrowWhenAdapterIsNull() { - assertThrows(NullPointerException.class, - () -> new GraphQueryOperations(null)); - } - } - - @Nested - @DisplayName("query() tests") - class QueryTests { - - @Test - @DisplayName("Should throw when pattern is null") - void shouldThrowWhenPatternIsNull() { - assertThrows(NullPointerException.class, - () -> queryOps.find(null)); - } - - @Test - @DisplayName("Should return stream of statements") - void shouldReturnStreamOfStatements() throws StorageException { - // Create a real mock statement - Statement mockStatement = mock(Statement.class); - - StatementPattern pattern = StatementPattern.of(null, null, null); - Set mockResults = new HashSet<>(Collections.singletonList(mockStatement)); - - // IMPORTANT: Return a NEW set each time (stream consumes the set) - when(mockAdapter.find(null, null, null, null)) - .thenAnswer(invocation -> new HashSet<>(mockResults)); - - Stream result = queryOps.find(pattern); - - assertNotNull(result); - } - - @Test - @DisplayName("Should wrap exceptions in StorageException") - void shouldWrapExceptionsInStorageException() { - StatementPattern pattern = StatementPattern.of(null, null, null); - when(mockAdapter.find(any(), any(), any(), any())) - .thenThrow(new RuntimeException("Test error")); - - assertThrows(StorageException.class, - () -> queryOps.find(pattern)); - } - } - - @Nested - @DisplayName("count() tests") - class CountTests { - - @Test - @DisplayName("Should return count of matching statements") - void shouldReturnCountOfMatchingStatements() throws StorageException { - // Create real mock statements - Statement mockStatement1 = mock(Statement.class); - Statement mockStatement2 = mock(Statement.class); - - StatementPattern pattern = StatementPattern.of(null, null, null); - Set mockResults = new HashSet<>(Arrays.asList(mockStatement1, mockStatement2)); - - // IMPORTANT: Return a NEW set each time - when(mockAdapter.find(null, null, null, null)) - .thenAnswer(invocation -> new HashSet<>(mockResults)); - - long count = queryOps.count(pattern); - - assertEquals(0, count); - } - - @Test - @DisplayName("Should return 0 for empty results") - void shouldReturnZeroForEmptyResults() throws StorageException { - StatementPattern pattern = StatementPattern.of(null, null, null); - - // Empty set works fine - when(mockAdapter.find(null, null, null, null)) - .thenReturn(Collections.emptySet()); - - long count = queryOps.count(pattern); - - assertEquals(0, count); - } - } - - @Nested - @DisplayName("exists() tests") - class ExistsTests { - - @Test - @DisplayName("Should return false when no statements exist") - void shouldReturnFalseWhenNoStatementsExist() throws StorageException { - StatementPattern pattern = StatementPattern.of(null, null, null); - - // Empty set works fine - when(mockAdapter.find(null, null, null, null)) - .thenReturn(Collections.emptySet()); - - boolean exists = queryOps.contains(pattern); - - assertFalse(exists); - } - } - -} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/graph/GraphStorageManagerTest.java b/src/test/java/fr/inria/corese/core/next/storage/impl/graph/GraphStorageManagerTest.java deleted file mode 100644 index deb55f45c..000000000 --- a/src/test/java/fr/inria/corese/core/next/storage/impl/graph/GraphStorageManagerTest.java +++ /dev/null @@ -1,211 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.graph; - -import fr.inria.corese.core.Graph; -import fr.inria.corese.core.next.data.api.factory.ValueFactory; -import fr.inria.corese.core.next.storage.api.lifecycle.LifecycleState; -import fr.inria.corese.core.next.storage.api.config.StorageConfig; -import fr.inria.corese.core.next.storage.api.transaction.TransactionManager; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.DisplayName; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; -import org.mockito.Mock; -import org.mockito.MockitoAnnotations; - -import static org.junit.jupiter.api.Assertions.*; -import static org.mockito.Mockito.verify; -import static org.mockito.Mockito.when; - -/** - * Unit tests for GraphStorageManager. - */ -class GraphStorageManagerTest { - - @Mock - private Graph mockGraph; - - @Mock - private ValueFactory mockFactory; - - private GraphStorageManager storageManager; - - @BeforeEach - void setUp() { - MockitoAnnotations.openMocks(this); - } - - @Nested - @DisplayName("Builder tests") - class BuilderTests { - - @Test - @DisplayName("Should build with graph and valueFactory") - void shouldBuildWithGraphAndValueFactory() { - storageManager = GraphStorageManager.builder() - .graph(mockGraph) - .valueFactory(mockFactory) - .build(); - - assertNotNull(storageManager); - assertEquals(mockGraph, storageManager.getGraph()); - } - - @Test - @DisplayName("Should throw when graph is null") - void shouldThrowWhenGraphIsNull() { - GraphStorageManager.Builder builder = GraphStorageManager.builder().valueFactory(mockFactory); - assertThrows(IllegalStateException.class, builder::build); - } - - @Test - @DisplayName("Should throw when valueFactory is null") - void shouldThrowWhenValueFactoryIsNull() { - GraphStorageManager.Builder builder = GraphStorageManager.builder().graph(mockGraph); - assertThrows(IllegalStateException.class, builder::build); - } - } - - @Nested - @DisplayName("Operations getters") - class OperationsGettersTests { - - @BeforeEach - void buildStorageManager() { - storageManager = GraphStorageManager.builder() - .graph(mockGraph) - .valueFactory(mockFactory) - .build(); - } - - @Test - @DisplayName("Should return QueryOperations") - void shouldReturnQueryOperations() { - assertNotNull(storageManager.queries()); - assertInstanceOf(GraphQueryOperations.class, - storageManager.queries()); - } - - @Test - @DisplayName("Should return MutationOperations") - void shouldReturnMutationOperations() { - assertNotNull(storageManager.mutations()); - assertInstanceOf(GraphMutationOperations.class, - storageManager.mutations()); - } - - @Test - @DisplayName("Should return MetadataOperations") - void shouldReturnMetadataOperations() { - assertNotNull(storageManager.metadata()); - assertInstanceOf(GraphMetadataOperations.class, - storageManager.metadata()); - } - - @Test - @DisplayName("Should return TransactionManager") - void shouldReturnTransactionManager() { - assertNotNull(storageManager.transactions()); - assertInstanceOf(GraphTransactionManager.class, - storageManager.transactions()); - } - - @Test - @DisplayName("Should return StorageLifecycle") - void shouldReturnStorageLifecycle() { - assertNotNull(storageManager.lifecycle()); - assertInstanceOf(GraphLifecycleManager.class, - storageManager.lifecycle()); - } - } - - @Nested - @DisplayName("Lifecycle tests") - class LifecycleTests { - - @BeforeEach - void buildStorageManager() { - storageManager = GraphStorageManager.builder() - .graph(mockGraph) - .valueFactory(mockFactory) - .build(); - } - - @Test - @DisplayName("Should start in NOT_INITIALIZED state") - void shouldStartInNotInitializedState() { - assertEquals(LifecycleState.NOT_INITIALIZED, - storageManager.lifecycle().getState()); - } - - @Test - @DisplayName("Should initialize successfully") - void shouldInitializeSuccessfully() { - StorageConfig config = StorageConfig.builder().build(); - - storageManager.lifecycle().initialize(config); - - assertEquals(LifecycleState.RUNNING, - storageManager.lifecycle().getState()); - verify(mockGraph).init(); - } - - @Test - @DisplayName("Should shutdown successfully") - void shouldShutdownSuccessfully() { - StorageConfig config = StorageConfig.builder().build(); - storageManager.lifecycle().initialize(config); - - storageManager.lifecycle().shutdown(); - - assertEquals(LifecycleState.SHUTDOWN, - storageManager.lifecycle().getState()); - } - } - - @Nested - @DisplayName("Transaction tests") - class TransactionTests { - - @BeforeEach - void buildStorageManager() { - storageManager = GraphStorageManager.builder() - .graph(mockGraph) - .valueFactory(mockFactory) - .build(); - } - - @Test - @DisplayName("Should not support transactions") - void shouldNotSupportTransactions() { - assertFalse(storageManager.transactions().supportsTransactions()); - } - - @Test - @DisplayName("Should throw when beginning transaction") - void shouldThrowWhenBeginningTransaction() { - TransactionManager txManager = storageManager.transactions(); - assertThrows(UnsupportedOperationException.class, txManager::beginTransaction); - } - } - - @Nested - @DisplayName("toString tests") - class ToStringTests { - - @Test - @DisplayName("Should return meaningful toString") - void shouldReturnMeaningfulToString() { - when(mockGraph.size()).thenReturn(42); - storageManager = GraphStorageManager.builder() - .graph(mockGraph) - .valueFactory(mockFactory) - .build(); - - String result = storageManager.toString(); - - assertNotNull(result); - assertTrue(result.contains("GraphStorageManager")); - assertTrue(result.contains("42")); - } - } -} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/graph/GraphTransactionManagerTest.java b/src/test/java/fr/inria/corese/core/next/storage/impl/graph/GraphTransactionManagerTest.java deleted file mode 100644 index 95618c30c..000000000 --- a/src/test/java/fr/inria/corese/core/next/storage/impl/graph/GraphTransactionManagerTest.java +++ /dev/null @@ -1,45 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.graph; - -import fr.inria.corese.core.next.storage.api.transaction.IsolationLevel; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.DisplayName; -import org.junit.jupiter.api.Test; - -import static org.junit.jupiter.api.Assertions.*; - -@DisplayName("GraphTransactionManager tests") -class GraphTransactionManagerTest { - - private GraphTransactionManager transactionManager; - - @BeforeEach - void setUp() { - transactionManager = new GraphTransactionManager(); - } - - @Test - @DisplayName("Should not support transactions") - void shouldNotSupportTransactions() { - assertFalse(transactionManager.supportsTransactions()); - } - - @Test - @DisplayName("Should throw when beginning transaction") - void shouldThrowWhenBeginningTransaction() { - assertThrows(UnsupportedOperationException.class, - () -> transactionManager.beginTransaction()); - } - - @Test - @DisplayName("Should throw when beginning transaction with isolation level") - void shouldThrowWhenBeginningTransactionWithIsolationLevel() { - assertThrows(UnsupportedOperationException.class, - () -> transactionManager.beginTransaction(IsolationLevel.READ_COMMITTED)); - } - - @Test - @DisplayName("Should return empty set of supported isolation levels") - void shouldReturnEmptySetOfSupportedIsolationLevels() { - assertTrue(transactionManager.getSupportedIsolationLevels().isEmpty()); - } -} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/CompactTermMapTest.java b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/CompactTermMapTest.java new file mode 100644 index 000000000..3df115c1b --- /dev/null +++ b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/CompactTermMapTest.java @@ -0,0 +1,78 @@ +package fr.inria.corese.core.next.storage.impl.memory; + +import fr.inria.corese.core.next.data.api.term.SimpleIRI; +import org.junit.jupiter.api.Test; + +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Random; +import java.util.concurrent.atomic.AtomicInteger; + +import static org.junit.jupiter.api.Assertions.*; + +class CompactTermMapTest { + private static TermKey key(int number) { + StringBuilder text = new StringBuilder("urn:"); + for (int i = 0; i < 5; i++) text.append((number & (1 << i)) == 0 ? "Aa" : "BB"); + return new TermKey(new SimpleIRI(text.toString())); + } + + private static List entries(CompactTermMap table) { + List result = new ArrayList<>(); + table.forEach((key, value) -> result.add(key.value().stringValue() + "=" + value)); + return result; + } + + @Test void collisionsAndRandomChangesMatchOrderedMapOracle() { + assertEquals(key(0).hashCode(), key(31).hashCode()); + var table = new CompactTermMap(); + var oracle = new LinkedHashMap(); + Random random = new Random(614615); + for (int operation = 0; operation < 3_000; operation++) { + TermKey key = key(random.nextInt(32)); + if (random.nextBoolean()) { + String value = "value" + operation; + assertEquals(oracle.putIfAbsent(key, value) == null, table.add(key, value)); + } else assertEquals(oracle.remove(key) != null, table.remove(key)); + assertEquals(oracle.size(), table.size()); + for (int i = 0; i < 32; i++) assertEquals(oracle.get(key(i)), table.get(key(i))); + assertEquals(new ArrayList<>(oracle.values()), entries(table).stream().map(s -> s.substring(s.indexOf('=') + 1)).toList()); + assertEquals(oracle.size(), table.sum(value -> 1)); + } + } + + @Test void copiesRemainIndependentThroughPromotionDemotionAndEmptying() { + var table = new CompactTermMap(); + for (int i = 0; i < 24; i++) { + table.add(key(i), "v" + i); + var snapshot = new CompactTermMap<>(table); + List expected = entries(snapshot); + table.add(key(31), "extra"); + assertEquals(expected, entries(snapshot)); + table.remove(key(31)); + } + var full = new CompactTermMap<>(table); + for (int i = 0; i < 24; i++) { + assertTrue(table.remove(key(i))); + var snapshot = new CompactTermMap<>(table); + List expected = entries(snapshot); + table.add(key(31), "extra"); + assertEquals(expected, entries(snapshot)); + table.remove(key(31)); + } + assertTrue(table.isEmpty()); + assertEquals(24, full.size()); + } + + @Test void shortCircuitVisitsOnlyTheFirstMatchingValue() { + var table = new CompactTermMap(); + for (int i = 0; i < 24; i++) { + table.add(key(i), "v" + i); + AtomicInteger calls = new AtomicInteger(); + assertTrue(table.visitMatching(null, value -> { calls.incrementAndGet(); return true; })); + assertEquals(1, calls.get()); + assertTrue(table.visitMatching(key(i), value -> value.equals("v" + (table.size() - 1)))); + } + } +} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/CowStructuralCopyTest.java b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/CowStructuralCopyTest.java new file mode 100644 index 000000000..47b5e6100 --- /dev/null +++ b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/CowStructuralCopyTest.java @@ -0,0 +1,167 @@ +package fr.inria.corese.core.next.storage.impl.memory; + +import fr.inria.corese.core.next.data.Values; +import fr.inria.corese.core.next.data.api.model.Statement; +import fr.inria.corese.core.next.data.api.term.SimpleIRI; +import fr.inria.corese.core.next.storage.Storages; +import fr.inria.corese.core.next.storage.api.StorageManager; +import fr.inria.corese.core.next.storage.api.config.StorageConfig; +import fr.inria.corese.core.next.storage.api.model.StatementPattern; +import org.junit.jupiter.api.Test; +import java.lang.reflect.Field; +import java.util.ArrayList; +import java.util.List; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNotEquals; +import static org.junit.jupiter.api.Assertions.assertNotNull; +import static org.junit.jupiter.api.Assertions.assertNotSame; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertSame; +import static org.junit.jupiter.api.Assertions.assertTrue; + +class CowStructuralCopyTest { + @Test + void mappedDenseCopyKeepsCollidingKeysHolesAndIndependentBuffers() throws ReflectiveOperationException { + var source = new DenseTermTable(); + for (int i = 0; i < 256; i++) { + source.put(DenseTermTableTest.key(i, true), new int[] { i }); + } + for (int i = 0; i < 256; i += 3) { + source.remove(DenseTermTableTest.key(i, true)); + } + var copy = new DenseTermTable<>(source, int[]::clone); + assertEquals(new ArrayList<>(source.keySet()), new ArrayList<>(copy.keySet())); + var oldKeys = source.keySet().iterator(); + for (var entry : copy.entrySet()) { + var key = entry.getKey(); + assertSame(oldKeys.next(), key); + assertNotSame(source.get(key), entry.getValue()); + entry.getValue()[0] = -1; + assertNotEquals(-1, source.get(key)[0]); + } + for (String name : List.of("keys", "data", "hashes", "positions")) { + Field field = storageField(DenseTermTable.class, name); + assertNotSame(field.get(source), field.get(copy)); + } + var removed = copy.keySet().iterator().next(); + assertNotNull(copy.remove(removed)); + assertNotNull(source.get(removed)); + for (int i = 300; i < 500; i++) { + copy.put(DenseTermTableTest.key(i, true), new int[] { i }); + } + assertNull(source.get(DenseTermTableTest.key(300, true))); + for (var entry : source.entrySet()) { + assertNotNull(source.get(entry.getKey())); + } + } + + @Test + void mappedCompactCopiesCoverSingletonPairsDenseAndRetainedDenseAfterChurn() throws ReflectiveOperationException { + for (int count : new int[] { 0, 1, 2, 8, 9, 40 }) { + assertCompactCopy(count); + } + } + + private static void assertCompactCopy(int count) throws ReflectiveOperationException { + var source = new CompactTermMap(); + for (int i = 0; i < count; i++) { + source.add(DenseTermTableTest.key(i, true), new int[] { i }); + } + if (count == 40) { + for (int i = 0; i < 34; i++) { + source.remove(DenseTermTableTest.key(i, true)); + } + } + List keys = new ArrayList<>(); + source.forEach((key, value) -> keys.add(key)); + var copy = new CompactTermMap<>(source, int[]::clone); + List copiedKeys = new ArrayList<>(); + copy.forEach((key, value) -> copiedKeys.add(key)); + assertEquals(keys, copiedKeys); + for (int i = 0; i < keys.size(); i++) { + assertSame(keys.get(i), copiedKeys.get(i)); + copy.get(keys.get(i))[0] = -1; + assertNotEquals(-1, source.get(keys.get(i))[0]); + } + Field entries = storageField(CompactTermMap.class, "entries"); + if (count > 1) { + assertNotSame(entries.get(source), entries.get(copy)); + } + copy.add(DenseTermTableTest.key(100, true), new int[] { 100 }); + for (var key : keys) { + copy.remove(key); + } + assertEquals(keys.size(), source.size()); + assertNull(source.get(DenseTermTableTest.key(100, true))); + } + + /** + * Inspects private storage buffers so the tests can assert COW container + * identity, which is deliberately unavailable through the production API. + */ + @SuppressWarnings("java:S3011") + private static Field storageField(Class type, String name) throws ReflectiveOperationException { + Field field = type.getDeclaredField(name); + field.setAccessible(true); + return field; + } + + @Test + void writerAndThreeReadersStayIsolatedAcrossDenseBranchLeafChangesAndRollback() { + var cfg = StorageConfig.memory(); + List input = createDenseStatements(); + try (var root = Storages.create(cfg); + var writer = root.openSession(); + var r1 = root.openSession(); + var r2 = root.openSession(); + var r3 = root.openSession()) { + root.mutations().addAll(input); + List stored; + try (var rows = root.queries().find(StatementPattern.matchAll())) { + stored = rows.toList(); + } + try (var t1 = r1.transactions().beginTransaction(); + var t2 = r2.transactions().beginTransaction(); + var t3 = r3.transactions().beginTransaction(); + var tx = writer.transactions().beginTransaction()) { + assertTrue(writer.mutations().remove(input.getFirst())); + assertEquals(143, writer.mutations().remove(StatementPattern.of(new SimpleIRI("urn:s0"), null, null))); + var fresh = Values.factory().createStatement(new SimpleIRI("urn:new"), new SimpleIRI("urn:new-p"), + new SimpleIRI("urn:new-o")); + assertTrue(writer.mutations().add(fresh)); + for (var reader : List.of(root, r1, r2, r3)) { + assertReaderSnapshot(reader, input, stored); + } + MemoryStorageAssertions.verify((IndexedStatementStore) writer.queries()); + tx.rollback(); + try (var rows = writer.queries().find(StatementPattern.matchAll())) { + assertEquals(input, rows.toList()); + } + } + } + } + + private static List createDenseStatements() { + List input = new ArrayList<>(); + for (int s = 0; s < 12; s++) { + for (int p = 0; p < 12; p++) { + for (int o = 0; o < 12; o++) { + input.add(Values.factory().createStatement(new SimpleIRI("urn:s" + s), new SimpleIRI("urn:p" + p), + new SimpleIRI("urn:o" + o))); + } + } + } + return input; + } + + private static void assertReaderSnapshot(StorageManager reader, List expected, List stored) { + try (var rows = reader.queries().find(StatementPattern.matchAll())) { + var actual = rows.toList(); + assertEquals(expected, actual); + for (int i = 0; i < actual.size(); i++) { + assertSame(stored.get(i), actual.get(i)); + } + } + MemoryStorageAssertions.verify((IndexedStatementStore) reader.queries()); + } +} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/DenseTermTableTest.java b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/DenseTermTableTest.java new file mode 100644 index 000000000..2e2633caa --- /dev/null +++ b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/DenseTermTableTest.java @@ -0,0 +1,133 @@ +package fr.inria.corese.core.next.storage.impl.memory; + +import fr.inria.corese.core.next.data.api.term.SimpleIRI; +import org.junit.jupiter.api.Test; +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Random; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertSame; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.junit.jupiter.api.Assertions.fail; + +class DenseTermTableTest { + static TermKey key(int i, boolean collision) { + StringBuilder s = new StringBuilder("urn:"); + if (collision) { + for (int b = 0; b < 9; b++) { + s.append((i & (1 << b)) == 0 ? "Aa" : "BB"); + } + } else { + s.append("key:").append(i); + } + return new TermKey(new SimpleIRI(s.toString())); + } + + @Test + void randomChangesAndCompactionPreserveOrderAndCopies() { + for (boolean collision : List.of(false, true)) { + exerciseRandomChanges(collision); + } + } + + private static void exerciseRandomChanges(boolean collision) { + var map = new DenseTermTable(); + var oracle = new LinkedHashMap(); + var random = new Random(17689); + for (int op = 0; op < 15000; op++) { + TermKey k = key(random.nextInt(512), collision); + String v = "v" + op; + switch (random.nextInt(5)) { + case 0 -> assertEquals(oracle.put(k, v), map.put(k, v)); + case 1 -> assertEquals(oracle.remove(k), map.remove(k)); + case 2 -> assertEquals(oracle.replace(k, v), map.replace(k, v)); + case 3 -> assertEquals(oracle.putIfAbsent(k, v), map.putIfAbsent(k, v)); + case 4 -> assertEquals(oracle.computeIfAbsent(k, x -> v), map.computeIfAbsent(k, x -> v)); + default -> throw new AssertionError("Unexpected random operation"); + } + assertEquals(oracle.get(k), map.get(k)); + assertEquals(oracle.size(), map.size()); + if (op % 71 == 0) { + assertEquals(new ArrayList<>(oracle.keySet()), new ArrayList<>(map.keySet())); + assertEquals(new ArrayList<>(oracle.values()), new ArrayList<>(map.values())); + var copy = new DenseTermTable<>(map); + map.put(key(513, false), "extra"); + assertEquals(new ArrayList<>(oracle.values()), new ArrayList<>(copy.values())); + map.remove(key(513, false)); + } + } + for (var k : new ArrayList<>(oracle.keySet())) { + assertEquals(oracle.remove(k), map.remove(k)); + } + assertTrue(map.isEmpty()); + map.put(key(0, collision), "reborn"); + assertEquals(List.of("reborn"), new ArrayList<>(map.values())); + } + + @Test + void duplicateFactoriesShortCircuitAndReinsertAppends() { + var map = new DenseTermTable(); + for (int i = 0; i < 400; i++) { + map.put(key(i, false), "v" + i); + } + assertEquals("v3", map.computeIfAbsent(key(3, false), k -> { + fail("existing key invokes factory"); + return null; + })); + for (int i = 0; i < 250; i++) { + map.remove(key(i, false)); + } + for (int i = 0; i < 250; i++) { + map.put(key(i, false), "v" + i); + } + assertEquals("v250", map.values().iterator().next()); + int[] calls = { 0 }; + assertTrue(map.visit(v -> { + calls[0]++; + return true; + })); + assertEquals(1, calls[0]); + assertEquals(400, map.sum(v -> 1)); + assertNull(map.computeIfAbsent(key(500, false), k -> null)); + assertEquals(400, map.size()); + } + + @Test + void bulkDeletionAndReinsertionPreserveCanonicalKeysAndSnapshots() { + var map = new DenseTermTable(); + var keys = new TermKey[1024]; + for (int i = 0; i < keys.length; i++) { + keys[i] = key(i, false); + map.put(keys[i], "v" + i); + } + var copy = new DenseTermTable<>(map); + for (int i = 0; i < 768; i++) { + assertEquals("v" + i, map.remove(keys[i])); + assertNull(map.get(keys[i])); + } + var survivors = map.keySet().iterator(); + for (int i = 768; i < 1024; i++) { + assertSame(keys[i], survivors.next()); + } + assertFalse(survivors.hasNext()); + for (int i = 0; i < 768; i++) { + map.put(keys[i], "v" + i); + } + var ordered = map.keySet().iterator(); + for (int i = 768; i < 1024; i++) { + assertSame(keys[i], ordered.next()); + } + for (int i = 0; i < 768; i++) { + assertSame(keys[i], ordered.next()); + } + assertFalse(ordered.hasNext()); + var snapshot = copy.keySet().iterator(); + for (var original : keys) { + assertSame(original, snapshot.next()); + } + assertFalse(snapshot.hasNext()); + } +} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/InMemoryStatementStoreTest.java b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/InMemoryStatementStoreTest.java deleted file mode 100644 index 119b8d25d..000000000 --- a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/InMemoryStatementStoreTest.java +++ /dev/null @@ -1,391 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.memory; - -import fr.inria.corese.core.next.data.api.term.*; -import fr.inria.corese.core.next.data.api.model.*; -import fr.inria.corese.core.next.data.api.factory.ValueFactory; -import fr.inria.corese.core.next.data.Values; -import org.junit.jupiter.api.*; - -import java.util.*; - -import static org.junit.jupiter.api.Assertions.*; -import static org.mockito.Mockito.*; - -/** - * Unit tests for InMemoryStatementStore. - */ -class InMemoryStatementStoreTest { - - private InMemoryStatementStore adapter; - private ValueFactory valueFactory; - - @BeforeEach - void setUp() { - adapter = new InMemoryStatementStore(); - valueFactory = Values.factory(); - } - - @Nested - @DisplayName("CRUD operations") - class CrudOperationsTests { - - @Test - @DisplayName("Should add statement successfully") - void shouldAddStatementSuccessfully() { - Statement stmt = mock(Statement.class); - - boolean added = adapter.add(stmt); - - assertTrue(added); - assertEquals(1, adapter.size()); - assertTrue(adapter.contains(stmt)); - } - - @Test - @DisplayName("Should not add duplicate statement") - void shouldNotAddDuplicateStatement() { - Statement stmt = mock(Statement.class); - adapter.add(stmt); - - boolean added = adapter.add(stmt); - - assertFalse(added); - assertEquals(1, adapter.size()); - } - - @Test - @DisplayName("Should remove statement successfully") - void shouldRemoveStatementSuccessfully() { - Statement stmt = mock(Statement.class); - adapter.add(stmt); - - boolean removed = adapter.remove(stmt); - - assertTrue(removed); - assertEquals(0, adapter.size()); - assertFalse(adapter.contains(stmt)); - } - - @Test - @DisplayName("Should return false when removing non-existent statement") - void shouldReturnFalseWhenRemovingNonExistentStatement() { - Statement stmt = mock(Statement.class); - - boolean removed = adapter.remove(stmt); - - assertFalse(removed); - } - - @Test - @DisplayName("Should check if statement exists") - void shouldCheckIfStatementExists() { - Statement stmt = mock(Statement.class); - - assertFalse(adapter.contains(stmt)); - - adapter.add(stmt); - assertTrue(adapter.contains(stmt)); - } - - @Test - @DisplayName("Should return correct size") - void shouldReturnCorrectSize() { - assertEquals(0, adapter.size()); - - adapter.add(mock(Statement.class)); - assertEquals(1, adapter.size()); - - adapter.add(mock(Statement.class)); - assertEquals(2, adapter.size()); - } - - @Test - @DisplayName("Should clear all statements") - void shouldClearAllStatements() { - adapter.add(mock(Statement.class)); - adapter.add(mock(Statement.class)); - adapter.add(mock(Statement.class)); - - adapter.clear(); - - assertEquals(0, adapter.size()); - } - } - - @Nested - @DisplayName("Pattern matching") - class PatternMatchingTests { - - @Test - @DisplayName("Should find all statements with null pattern") - void shouldFindAllStatementsWithNullPattern() { - Statement stmt1 = mock(Statement.class); - Statement stmt2 = mock(Statement.class); - adapter.add(stmt1); - adapter.add(stmt2); - - Set results = adapter.find(null, null, null, null); - - assertEquals(2, results.size()); - assertTrue(results.contains(stmt1)); - assertTrue(results.contains(stmt2)); - } - - @Test - @DisplayName("Should find statements by subject") - void shouldFindStatementsBySubject() { - Resource subject1 = mock(Resource.class); - Resource subject2 = mock(Resource.class); - IRI predicate = mock(IRI.class); - Value object = mock(Value.class); - - Statement stmt1 = mock(Statement.class); - when(stmt1.getSubject()).thenReturn(subject1); - when(stmt1.getPredicate()).thenReturn(predicate); - when(stmt1.getObject()).thenReturn(object); - - Statement stmt2 = mock(Statement.class); - when(stmt2.getSubject()).thenReturn(subject2); - when(stmt2.getPredicate()).thenReturn(predicate); - when(stmt2.getObject()).thenReturn(object); - - adapter.add(stmt1); - adapter.add(stmt2); - - Set results = adapter.find(subject1, null, null, null); - - assertEquals(1, results.size()); - assertTrue(results.contains(stmt1)); - } - - @Test - @DisplayName("Should find statements by predicate") - void shouldFindStatementsByPredicate() { - Resource subject = mock(Resource.class); - IRI predicate1 = mock(IRI.class); - IRI predicate2 = mock(IRI.class); - Value object = mock(Value.class); - - Statement stmt1 = mock(Statement.class); - when(stmt1.getSubject()).thenReturn(subject); - when(stmt1.getPredicate()).thenReturn(predicate1); - when(stmt1.getObject()).thenReturn(object); - - Statement stmt2 = mock(Statement.class); - when(stmt2.getSubject()).thenReturn(subject); - when(stmt2.getPredicate()).thenReturn(predicate2); - when(stmt2.getObject()).thenReturn(object); - - adapter.add(stmt1); - adapter.add(stmt2); - - Set results = adapter.find(null, predicate1, null, null); - - assertEquals(1, results.size()); - assertTrue(results.contains(stmt1)); - } - - @Test - @DisplayName("Should find statements by context") - void shouldFindStatementsByContext() { - Resource subject = valueFactory.createIRI("http://example.org/s"); - IRI predicate = valueFactory.createIRI("http://example.org/p"); - Value object = valueFactory.createLiteral("value"); - Resource context1 = valueFactory.createIRI("http://example.org/graph1"); - Resource equivalentContext1 = valueFactory.createIRI("http://example.org/graph1"); - Resource context2 = valueFactory.createIRI("http://example.org/graph2"); - - Statement stmt1 = valueFactory.createStatement(subject, predicate, object, context1); - Statement stmt2 = valueFactory.createStatement(subject, predicate, object, context2); - - adapter.add(stmt1); - adapter.add(stmt2); - - Set results = adapter.find(null, null, null, new Resource[]{equivalentContext1}); - - assertEquals(1, results.size()); - assertTrue(results.contains(stmt1)); - } - - @Test - @DisplayName("Should not match IRI and blank node contexts sharing the same string value") - void shouldNotMatchIriAndBlankNodeContextsWithSameStringValue() { - Resource subject = valueFactory.createIRI("http://example.org/s"); - IRI predicate = valueFactory.createIRI("http://example.org/p"); - Value object = valueFactory.createLiteral("value"); - BNode blankNodeContext = valueFactory.createBNode("http://example.org/g"); - IRI iriContext = valueFactory.createIRI("http://example.org/g"); - - Statement stmt = valueFactory.createStatement(subject, predicate, object, blankNodeContext); - adapter.add(stmt); - - Set results = adapter.find(null, null, null, new Resource[]{iriContext}); - - assertTrue(results.isEmpty()); - } - - @Test - @DisplayName("Should return empty set when no matches") - void shouldReturnEmptySetWhenNoMatches() { - adapter.add(mock(Statement.class)); - - Resource unknownSubject = mock(Resource.class); - Set results = adapter.find(unknownSubject, null, null, null); - - assertTrue(results.isEmpty()); - } - } - - @Nested - @DisplayName("Context operations") - class ContextOperationsTests { - - @Test - @DisplayName("Should clear specific context") - void shouldClearSpecificContext() { - Resource context1 = mock(Resource.class); - Resource context2 = mock(Resource.class); - - Statement stmt1 = mock(Statement.class); - when(stmt1.getContext()).thenReturn(context1); - - Statement stmt2 = mock(Statement.class); - when(stmt2.getContext()).thenReturn(context2); - - adapter.add(stmt1); - adapter.add(stmt2); - - adapter.clearContext(context1); - - assertEquals(1, adapter.size()); - assertFalse(adapter.contains(stmt1)); - assertTrue(adapter.contains(stmt2)); - } - - @Test - @DisplayName("Should handle clearing non-existent context") - void shouldHandleClearingNonExistentContext() { - adapter.add(mock(Statement.class)); - Resource unknownContext = mock(Resource.class); - - adapter.clearContext(unknownContext); - - assertEquals(1, adapter.size()); - } - } - - @Nested - @DisplayName("Metadata operations") - class MetadataOperationsTests { - - @Test - @DisplayName("Should return all subjects") - void shouldReturnAllSubjects() { - Resource subject1 = mock(Resource.class); - Resource subject2 = mock(Resource.class); - IRI predicate = mock(IRI.class); - Value object = mock(Value.class); - - Statement stmt1 = mock(Statement.class); - when(stmt1.getSubject()).thenReturn(subject1); - when(stmt1.getPredicate()).thenReturn(predicate); - when(stmt1.getObject()).thenReturn(object); - - Statement stmt2 = mock(Statement.class); - when(stmt2.getSubject()).thenReturn(subject2); - when(stmt2.getPredicate()).thenReturn(predicate); - when(stmt2.getObject()).thenReturn(object); - - adapter.add(stmt1); - adapter.add(stmt2); - - Set subjects = adapter.getSubjects(); - - assertEquals(2, subjects.size()); - assertTrue(subjects.contains(subject1)); - assertTrue(subjects.contains(subject2)); - } - - @Test - @DisplayName("Should return all predicates") - void shouldReturnAllPredicates() { - Resource subject = mock(Resource.class); - IRI predicate1 = mock(IRI.class); - IRI predicate2 = mock(IRI.class); - Value object = mock(Value.class); - - Statement stmt1 = mock(Statement.class); - when(stmt1.getSubject()).thenReturn(subject); - when(stmt1.getPredicate()).thenReturn(predicate1); - when(stmt1.getObject()).thenReturn(object); - - Statement stmt2 = mock(Statement.class); - when(stmt2.getSubject()).thenReturn(subject); - when(stmt2.getPredicate()).thenReturn(predicate2); - when(stmt2.getObject()).thenReturn(object); - - adapter.add(stmt1); - adapter.add(stmt2); - - Set predicates = adapter.getPredicates(); - - assertEquals(2, predicates.size()); - assertTrue(predicates.contains(predicate1)); - assertTrue(predicates.contains(predicate2)); - } - - @Test - @DisplayName("Should return all objects") - void shouldReturnAllObjects() { - Resource subject = mock(Resource.class); - IRI predicate = mock(IRI.class); - Value object1 = mock(Value.class); - Value object2 = mock(Value.class); - - Statement stmt1 = mock(Statement.class); - when(stmt1.getSubject()).thenReturn(subject); - when(stmt1.getPredicate()).thenReturn(predicate); - when(stmt1.getObject()).thenReturn(object1); - - Statement stmt2 = mock(Statement.class); - when(stmt2.getSubject()).thenReturn(subject); - when(stmt2.getPredicate()).thenReturn(predicate); - when(stmt2.getObject()).thenReturn(object2); - - adapter.add(stmt1); - adapter.add(stmt2); - - Set objects = adapter.getObjects(); - - assertEquals(2, objects.size()); - assertTrue(objects.contains(object1)); - assertTrue(objects.contains(object2)); - } - - @Test - @DisplayName("Should return all contexts excluding null") - void shouldReturnAllContextsExcludingNull() { - Resource context1 = mock(Resource.class); - Resource context2 = mock(Resource.class); - - Statement stmt1 = mock(Statement.class); - when(stmt1.getContext()).thenReturn(context1); - - Statement stmt2 = mock(Statement.class); - when(stmt2.getContext()).thenReturn(context2); - - Statement stmt3 = mock(Statement.class); - when(stmt3.getContext()).thenReturn(null); // Default graph - - adapter.add(stmt1); - adapter.add(stmt2); - adapter.add(stmt3); - - Set contexts = adapter.getContexts(); - - assertEquals(2, contexts.size()); - assertTrue(contexts.contains(context1)); - assertTrue(contexts.contains(context2)); - } - } -} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/IngestionAllocationTest.java b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/IngestionAllocationTest.java new file mode 100644 index 000000000..fe03b6253 --- /dev/null +++ b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/IngestionAllocationTest.java @@ -0,0 +1,151 @@ +package fr.inria.corese.core.next.storage.impl.memory; + +import fr.inria.corese.core.next.data.Values; +import fr.inria.corese.core.next.data.api.model.Statement; +import fr.inria.corese.core.next.data.api.term.SimpleBNode; +import fr.inria.corese.core.next.data.api.term.SimpleIRI; +import fr.inria.corese.core.next.data.api.term.SimpleLiteral; +import fr.inria.corese.core.next.data.api.term.SimpleTriple; +import fr.inria.corese.core.next.data.api.term.Value; +import fr.inria.corese.core.next.storage.Storages; +import fr.inria.corese.core.next.storage.api.config.StorageConfig; +import fr.inria.corese.core.next.storage.api.exception.StorageException; +import fr.inria.corese.core.next.storage.api.model.StatementPattern; +import org.junit.jupiter.api.Test; +import java.util.AbstractCollection; +import java.util.ArrayList; +import java.util.Iterator; +import java.util.List; +import java.util.concurrent.atomic.AtomicInteger; +import static org.junit.jupiter.api.Assertions.*; + +class IngestionAllocationTest { + private static final StorageConfig CONFIG = StorageConfig.memory(); + + @Test void rawLookupMatchesKeyLookupForKindsSplitsLexicalFormsAndCollisions() { + var table = new DenseTermTable(); + var datatype = new SimpleIRI("http://www.w3.org/2001/XMLSchema#integer"); + List values = new ArrayList<>(List.of(new SimpleIRI("urn:same"), new SimpleBNode("urn:same"), + new SimpleLiteral("01", datatype), new SimpleLiteral("1", datatype), + new SimpleLiteral("word", "en"), new SimpleLiteral("word", "fr"), + new SimpleIRI("http://example/a", "bc"), + new SimpleTriple(new SimpleIRI("urn:s"), new SimpleIRI("urn:p"), new SimpleLiteral("x")))); + for (int i = 0; i < 512; i++) values.add(DenseTermTableTest.key(i, true).value()); + for (int i = 0; i < values.size(); i++) table.put(new TermKey(values.get(i)), i); + for (Value value : values) assertEquals(table.get(new TermKey(value)), table.get(value, TermKey.structuralHash(value))); + assertEquals(6, table.get(new SimpleIRI("http://example/ab", "c"), TermKey.structuralHash(values.get(6)))); + var miss = new SimpleIRI("urn:absent"); + assertNull(table.get(miss, TermKey.structuralHash(miss))); + assertNull(table.get(values.getFirst(), TermKey.structuralHash(values.getFirst()) ^ 1)); + for (int i = 0; i < 100; i++) table.remove(new TermKey(values.get(i))); + for (Value value : values) assertEquals(table.get(new TermKey(value)), table.get(value, TermKey.structuralHash(value))); + } + + @Test void stagingProbeNeverBecomesAKeyAndCopiesKeepCanonicalIdentity() { + var terms = new StagingTerms(4); + var value = new SimpleIRI("http://example/a", "bc"); + var key = new TermKey(value); + terms.put(key, key); + assertSame(key, terms.intern(new SimpleIRI("http://example/ab", "c"))); + assertNull(terms.find(new SimpleBNode("http://example/abc"), key.hashCode())); + var copy = new StagingTerms(terms); + for (int i = 0; i < 512; i++) { + TermKey collision = DenseTermTableTest.key(i, true); + terms.put(collision, collision); + assertSame(collision, terms.intern(collision.value())); + } + assertEquals(513, terms.size()); assertEquals(1, copy.size()); + assertNull(copy.find(DenseTermTableTest.key(100, true).value(), DenseTermTableTest.key(100, true).hashCode())); + var added = new SimpleIRI("urn:copy-only"); + copy.intern(added); + assertNull(terms.find(added, TermKey.structuralHash(added))); + assertSame(key, terms.find(value, key.hashCode())); + terms.forEach((stored, canonical) -> { assertEquals(TermKey.class, stored.getClass()); assertSame(stored, canonical); }); + } + + @Test void sizedStreamingCollectionCreatesMultipleGraphsWithSnapshotIsolation() { + List input = new ArrayList<>(); + var predicate = new SimpleIRI("urn:p"); + for (int i = 0; i < 1024; i++) input.add(Values.factory().createStatement(new SimpleIRI("urn:s" + i), + predicate, new SimpleIRI("urn:o" + i), new SimpleIRI("urn:g" + i % 128))); + var iterations = new AtomicInteger(); + var sizes = new AtomicInteger(); + var source = new AbstractCollection() { + @Override public int size() { sizes.incrementAndGet(); return input.size(); } + @Override public Iterator iterator() { + assertEquals(1, iterations.incrementAndGet()); return input.iterator(); + } + }; + try (var root = Storages.create(CONFIG); var reader = root.openSession(); + var snapshot = reader.transactions().beginTransaction()) { + assertEquals(1024, root.mutations().addAll(source)); + assertEquals(128, root.metadata().getContexts().size()); + assertEquals(0, reader.queries().count(StatementPattern.matchAll())); + assertEquals(1024, root.queries().count(StatementPattern.matchAll())); + MemoryStorageAssertions.verify((IndexedStatementStore) root.queries()); + } + assertEquals(1, iterations.get()); assertEquals(1, sizes.get()); + } + + @Test void collectionWithUnavailableSizeIsConsumedOnceAndPublishedAtomically() { + var iri = new SimpleIRI("urn:shared-role"); + List input = new ArrayList<>(); + for (int i = 0; i < 1024; i++) input.add(Values.factory().createStatement(iri, iri, new SimpleIRI("urn:o" + i))); + var iterations = new AtomicInteger(); + var source = new AbstractCollection() { + @Override public int size() { throw new UnsupportedOperationException("streaming collection"); } + @Override public Iterator iterator() { + assertEquals(1, iterations.incrementAndGet()); return input.iterator(); + } + }; + try (var root = Storages.create(CONFIG); var reader = root.openSession()) { + root.mutations().add(input.getFirst()); + try (var tx = reader.transactions().beginTransaction()) { + assertEquals(1023, root.mutations().addAll(source)); + assertEquals(1, reader.queries().count(StatementPattern.matchAll())); + assertEquals(1024, root.queries().count(StatementPattern.matchAll())); + MemoryStorageAssertions.verify((IndexedStatementStore) root.queries()); + MemoryStorageAssertions.verify((IndexedStatementStore) reader.queries()); + try (var rows = root.queries().find(StatementPattern.matchAll())) { assertEquals(input, rows.toList()); } + } + } + assertEquals(1, iterations.get()); + } + + @Test void failedStreamingBatchPreservesPriorWritesAndOtherSnapshots() { + var s = new SimpleIRI("urn:s"); + var p = new SimpleIRI("urn:p"); + var old = Values.factory().createStatement(s, p, new SimpleIRI("urn:old")); + var prior = Values.factory().createStatement(s, p, new SimpleIRI("urn:prior")); + try (var root = Storages.create(CONFIG); var writer = root.openSession(); var reader = root.openSession()) { + root.mutations().add(old); + try (var snapshot = reader.transactions().beginTransaction(); var tx = writer.transactions().beginTransaction()) { + writer.mutations().add(prior); + var iterations = new AtomicInteger(); + Iterable failing = () -> { + assertEquals(1, iterations.incrementAndGet()); + return new Iterator<>() { + int index; + public boolean hasNext() { + if (index == 256) { + throw new IllegalArgumentException("input failure"); + } + return true; + } + public Statement next() { return Values.factory().createStatement(s, p, new SimpleIRI("urn:new" + index++)); } + }; + }; + var failure = assertThrows(StorageException.class, () -> writer.mutations().addAll(failing)); + assertInstanceOf(IllegalArgumentException.class, failure.getCause()); + try (var rows = writer.queries().find(StatementPattern.matchAll())) { assertEquals(List.of(old, prior), rows.toList()); } + assertEquals(1, root.queries().count(StatementPattern.matchAll())); + assertEquals(1, reader.queries().count(StatementPattern.matchAll())); + MemoryStorageAssertions.verify((IndexedStatementStore) writer.queries()); + tx.commit(); + assertEquals(2, root.queries().count(StatementPattern.matchAll())); + assertEquals(1, reader.queries().count(StatementPattern.matchAll())); + assertEquals(1, iterations.get()); + } + } + } +} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/LazyProducerTest.java b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/LazyProducerTest.java new file mode 100644 index 000000000..db8280e3d --- /dev/null +++ b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/LazyProducerTest.java @@ -0,0 +1,228 @@ +package fr.inria.corese.core.next.storage.impl.memory; + +import fr.inria.corese.core.next.data.Values; +import fr.inria.corese.core.next.data.api.model.Statement; +import fr.inria.corese.core.next.data.api.term.SimpleIRI; +import fr.inria.corese.core.next.query.api.exception.QueryEvaluationException; +import fr.inria.corese.core.next.query.api.exception.QueryTimeoutException; +import fr.inria.corese.core.next.query.impl.engine.eval.Eval; +import fr.inria.corese.core.next.query.impl.engine.eval.RdfTermMatcher; +import fr.inria.corese.core.next.query.impl.engine.eval.SparqlKgramEvaluator; +import fr.inria.corese.core.next.query.impl.engine.event.ResultListener; +import fr.inria.corese.core.next.query.impl.engine.model.Edge; +import fr.inria.corese.core.next.query.impl.engine.model.Node; +import fr.inria.corese.core.next.query.impl.engine.pattern.Exp; +import fr.inria.corese.core.next.query.impl.engine.spi.Environment; +import fr.inria.corese.core.next.query.impl.engine.storage.StorageManagerProducer; +import fr.inria.corese.core.next.query.impl.sparql.bridge.CoreseAstQueryBuilder; +import fr.inria.corese.core.next.query.impl.sparql.execution.NextSparqlPipelineExecutor; +import fr.inria.corese.core.next.query.impl.sparql.parser.SparqlParser; +import fr.inria.corese.core.next.query.impl.sparql.ast.SelectQueryAst; +import fr.inria.corese.core.next.storage.Storages; +import fr.inria.corese.core.next.storage.api.StorageManager; +import fr.inria.corese.core.next.storage.api.config.StorageConfig; +import fr.inria.corese.core.next.storage.api.model.StatementPattern; +import fr.inria.corese.core.next.storage.api.operations.QueryOperations; +import org.junit.jupiter.api.Test; + +import java.util.ArrayList; +import java.util.List; +import java.util.Map; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicInteger; +import java.util.function.Consumer; +import java.util.stream.Stream; + +import static org.junit.jupiter.api.Assertions.*; +import static org.mockito.ArgumentMatchers.*; +import static org.mockito.Mockito.*; + +class LazyProducerTest { + private static final String PREFIX = "PREFIX : "; + + private static final class Fixture implements AutoCloseable { + final StorageManager root = Storages.create(StorageConfig.memory()); + final StorageManager observed = mock(StorageManager.class); + final QueryOperations queries = mock(QueryOperations.class); + final AtomicInteger opened = new AtomicInteger(); + final AtomicInteger read = new AtomicInteger(); + final AtomicInteger closed = new AtomicInteger(); + Consumer callback = s -> { }; + final NextSparqlPipelineExecutor executor; + + Fixture() { + root.mutations().addAll(StorageBenchmark.data(20, 0, false)); + when(observed.queries()).thenReturn(queries); + when(observed.metadata()).thenReturn(root.metadata()); + when(queries.find(any())).thenAnswer(call -> { + opened.incrementAndGet(); + return root.queries().find(call.getArgument(0)).peek(s -> { + read.incrementAndGet(); callback.accept(s); + }).onClose(closed::incrementAndGet); + }); + executor = new NextSparqlPipelineExecutor(observed); + } + + @SuppressWarnings("java:S3011") + Object partition() throws ReflectiveOperationException { + Object store = root.queries(); + var method = store.getClass().getDeclaredMethod("state"); + method.setAccessible(true); + Object state = method.invoke(store); + var field = state.getClass().getDeclaredField("graphs"); + field.setAccessible(true); + return ((Map) field.get(state)).get(null); + } + + @SuppressWarnings("java:S3011") + int references() throws ReflectiveOperationException { + Object partition = partition(); + var field = partition.getClass().getDeclaredField("references"); + field.setAccessible(true); + return field.getInt(partition); + } + void assertReleased() throws Exception { + assertEquals(opened.get(), closed.get(), "all opened streams closed exactly once"); + assertEquals(1, references(), "only the live storage state owns the partition"); + Object partition = partition(); + root.mutations().add(Values.factory().createStatement(new SimpleIRI("urn:new:s"), + new SimpleIRI("urn:new:p"), new SimpleIRI("urn:new:o"))); + assertSame(partition, partition(), "no ghost snapshot forces the next mutation to copy"); + } + @Override public void close() { root.close(); } + } + + @Test void askAndLimitsReadOnlyTheRequestedPrefixAndReleaseSnapshots() throws Exception { + for (int limit : new int[]{0, 1, 10}) try (var f = new Fixture()) { + if (limit == 0) assertTrue(f.executor.evaluateBoolean("ASK { ?s ?p ?o }")); + else try (var result = f.executor.evaluateTuple("SELECT * { ?s ?p ?o } LIMIT " + limit)) { + assertEquals(limit, result.stream().count()); + } + assertEquals(limit == 0 ? 1 : limit, f.read.get()); + f.assertReleased(); + } + } + + @Test void selectiveBgpPushesBindingsWithoutBuildingAllFirstEdgeSolutions() throws Exception { + try (var f = new Fixture(); var result = f.executor.evaluateTuple(PREFIX + + "SELECT ?s ?name { ?s :type :Person . ?s :name ?name } LIMIT 1")) { + assertTrue(result.hasNext()); + var row = result.next(); assertEquals("urn:bench:s0", row.getValue("s").stringValue()); + assertEquals("name0", row.getValue("name").stringValue()); assertFalse(result.hasNext()); + assertEquals(2, f.read.get()); f.assertReleased(); + } + } + + @Test void defaultGraphDeduplicatesWhileNamedGraphsKeepTheirOwnSolutions() throws Exception { + try (var f = new Fixture()) { + var duplicate = Values.factory().createStatement(new SimpleIRI("urn:bench:s0"), + new SimpleIRI("urn:bench:name"), Values.factory().createLiteral("name0"), new SimpleIRI("urn:graph")); + f.root.mutations().add(duplicate); + try (var result = f.executor.evaluateTuple(PREFIX + "SELECT ?s ?name { ?s :name ?name }")) { + assertEquals(20, result.stream().count()); + } + try (var result = f.executor.evaluateTuple(PREFIX + "SELECT ?g ?s { GRAPH ?g { ?s :name ?name } }")) { + assertEquals(1, result.stream().count()); + } + f.assertReleased(); + } + } + + @Test void fullScansEmptyMatchesOffsetDistinctAndOrderingPreserveSemantics() throws Exception { + try (var f = new Fixture()) { + try (var result = f.executor.evaluateTuple("SELECT * { ?s ?p ?o }")) { + assertEquals(120, result.stream().count()); + } + assertFalse(f.executor.evaluateBoolean("ASK { ?p ?o }")); + try (var result = f.executor.evaluateTuple(PREFIX + + "SELECT DISTINCT ?g { ?s :group ?g } ORDER BY DESC(?g) OFFSET 1 LIMIT 2")) { + assertEquals(List.of("urn:bench:g8", "urn:bench:g7"), result.stream() + .map(b -> b.getValue("g").stringValue()).toList()); + } + f.assertReleased(); + } + } + + @Test void askRespectsOffsetLimitZeroAndOrdering() throws Exception { + try (var f = new Fixture()) { + assertTrue(f.executor.evaluateBoolean("ASK { ?s ?p ?o } OFFSET 119")); + assertFalse(f.executor.evaluateBoolean("ASK { ?s ?p ?o } OFFSET 120")); + assertFalse(f.executor.evaluateBoolean("ASK { ?s ?p ?o } LIMIT 0")); + assertTrue(f.executor.evaluateBoolean("ASK { ?s ?p ?o } ORDER BY DESC(?s) OFFSET 119")); + f.assertReleased(); + } + } + + @Test void storageTraversalFailureReleasesSnapshot() throws Exception { + try (var f = new Fixture()) { + f.callback = s -> { throw new IllegalStateException("storage traversal failed"); }; + assertThrows(IllegalStateException.class, () -> f.executor.evaluateBoolean("ASK { ?s ?p ?o }")); + f.assertReleased(); + } + } + + @Test void matcherOrListenerFailureClosesTheCurrentlySuspendedEdgeCursor() throws Exception { + try (var f = new Fixture()) { + var producer = new StorageManagerProducer(f.observed); + var eval = Eval.create(producer, new SparqlKgramEvaluator(), new RdfTermMatcher()); + var query = new CoreseAstQueryBuilder().toNextQuery((SelectQueryAst) new SparqlParser().parse("SELECT * { ?s ?p ?o }")); + ResultListener listener = mock(ResultListener.class); + when(listener.listen(any(Exp.class), anyInt())).thenAnswer(call -> call.getArgument(0)); + when(listener.listen(any(Exp.class), any(Edge.class), any(Edge.class))) + .thenThrow(new IllegalStateException("listener failed")); + eval.setListener(listener); + assertThrows(IllegalStateException.class, () -> eval.query(query)); + assertEquals(1, f.read.get()); f.assertReleased(); + } + } + + @Test void iterablesAreLazyRepeatableAndCloseIdempotently() throws Exception { + try (var f = new Fixture()) { + var producer = new StorageManagerProducer(f.observed); + var query = new CoreseAstQueryBuilder().toNextQuery((SelectQueryAst) new SparqlParser().parse("SELECT * { ?s ?p ?o }")); + Edge edge = query.getBody().first().first().getEdge(); + Environment environment = mock(Environment.class); when(environment.getQuery()).thenReturn(query); + var iterable = producer.getEdges(null, List.of(), edge, environment); + assertEquals(0, f.opened.get()); + var iterator = iterable.iterator(); assertEquals(2, f.references()); + assertTrue(iterator.hasNext()); assertTrue(iterator.hasNext()); assertEquals(1, f.read.get()); + assertNotNull(iterator.next()); producer.closeEdges(iterable); producer.closeEdges(iterable); + assertFalse(iterator.hasNext()); + List rows = new ArrayList<>(); iterable.forEach(rows::add); assertEquals(120, rows.size()); + producer.finish(query); producer.close(); producer.close(); f.assertReleased(); + } + } + + @Test void interruptionDuringTraversalClosesTheCursorAndPreservesInterruptStatus() throws Exception { + try (var f = new Fixture()) { + f.callback = s -> Thread.currentThread().interrupt(); + try { + assertThrows(QueryEvaluationException.class, () -> f.executor.evaluateTuple("SELECT * { ?s ?p ?o }")); + assertTrue(Thread.currentThread().isInterrupted()); + } finally { Thread.interrupted(); } + f.assertReleased(); + } + } + + @Test void timeoutClosesAnActuallyOpenSnapshotBeforeReturningToTheCaller() throws Exception { + try (var f = new Fixture()) { + // Warm the parser/bridge before installing a deterministically blocking source. + assertTrue(f.executor.evaluateBoolean("ASK { ?s ?p ?o }")); + CountDownLatch entered = new CountDownLatch(1); + f.callback = s -> { + entered.countDown(); + try { new CountDownLatch(1).await(); } + catch (InterruptedException failure) { + Thread.currentThread().interrupt(); + throw new QueryEvaluationException("interrupted source", failure); + } + }; + var result = CompletableFuture.runAsync(() -> assertThrows(QueryTimeoutException.class, + () -> f.executor.evaluateTuple("SELECT * { ?s ?p ?o }", null, null, 500))); + assertTrue(entered.await(5, TimeUnit.SECONDS), "cursor opened before timeout"); + result.get(5, TimeUnit.SECONDS); f.assertReleased(); + } + } +} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/LazyReadTest.java b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/LazyReadTest.java new file mode 100644 index 000000000..526313866 --- /dev/null +++ b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/LazyReadTest.java @@ -0,0 +1,260 @@ +package fr.inria.corese.core.next.storage.impl.memory; + +import fr.inria.corese.core.next.data.Values; +import fr.inria.corese.core.next.data.api.model.Statement; +import fr.inria.corese.core.next.data.api.term.Resource; +import fr.inria.corese.core.next.data.api.term.SimpleIRI; +import fr.inria.corese.core.next.storage.Storages; +import fr.inria.corese.core.next.storage.api.StorageManager; +import fr.inria.corese.core.next.storage.api.config.StorageConfig; +import fr.inria.corese.core.next.storage.api.model.StatementPattern; +import org.junit.jupiter.api.Test; +import java.lang.reflect.Field; +import java.util.ArrayList; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.CountDownLatch; +import static org.junit.jupiter.api.Assertions.*; + +class LazyReadTest { + private static final Resource G = new SimpleIRI("urn:lazy:g"); + private static final Resource H = new SimpleIRI("urn:lazy:h"); + private static StorageManager storage() { return Storages.create(StorageConfig.memory()); } + private static Statement row(int s, int p, int o, Resource g) { + return Values.factory().createStatement(new SimpleIRI("urn:lazy:s" + s), + new SimpleIRI("urn:lazy:p" + p), new SimpleIRI("urn:lazy:o" + o), g); + } + @SuppressWarnings("java:S3011") + private static Object partition(StorageManager storage, Resource graph) throws ReflectiveOperationException { + Object store = storage.queries(); + var method = store.getClass().getDeclaredMethod("state"); + method.setAccessible(true); + Object state = method.invoke(store); + Field field = state.getClass().getDeclaredField("graphs"); + field.setAccessible(true); + return ((Map) field.get(state)).get(graph); + } + @SuppressWarnings("java:S3011") + private static int references(Object partition) throws ReflectiveOperationException { + Field field = partition.getClass().getDeclaredField("references"); + field.setAccessible(true); + return field.getInt(partition); + } + private static List all(StorageManager storage) { + try (var stream = storage.queries().find(StatementPattern.matchAll())) { return stream.toList(); } + } + + @Test void positionalCursorMatchesBulkTraversalAcrossAllMasksDenseLeavesAndHoles() { + try (var storage = storage()) { + List surviving = createSurvivingDataset(storage); + for (int mask = 0; mask < 8; mask++) { + for (Resource[] graphs : new Resource[][]{{}, {null}, {H, G}, {G, H}}) { + var pattern = StatementPattern.of((mask & 4) == 0 ? null : new SimpleIRI("urn:lazy:s2"), + (mask & 2) == 0 ? null : new SimpleIRI("urn:lazy:p3"), + (mask & 1) == 0 ? null : new SimpleIRI("urn:lazy:o4"), graphs); + verifyPatternTraversal(storage, surviving, pattern, mask); + } + } + MemoryStorageAssertions.verify((IndexedStatementStore) storage.queries()); + } + } + + private static List createSurvivingDataset(StorageManager storage) { + List input = new ArrayList<>(); + for (Resource g : new Resource[]{null, G, H}) { + for (int s = 0; s < 12; s++) { + for (int p = 0; p < 12; p++) { + for (int o = 0; o < 12; o++) { + input.add(row(s, p, o, g)); + } + } + } + } + storage.mutations().addAll(input); + for (int i = 0; i < input.size(); i += 7) { + storage.mutations().remove(input.get(i)); + } + return all(storage); + } + + private static void verifyPatternTraversal(StorageManager storage, List surviving, + StatementPattern pattern, int mask) { + List bulk; + List cursor = new ArrayList<>(); + try (var stream = storage.queries().find(pattern)) { bulk = stream.toList(); } + try (var stream = storage.queries().find(pattern)) { + var iterator = stream.iterator(); + while (iterator.hasNext()) { + assertTrue(iterator.hasNext()); + cursor.add(iterator.next()); + } + } + assertEquals(bulk, cursor, "encounter order, mask " + mask); + var oracle = surviving.stream().filter(s -> matchesPattern(s, pattern)).toList(); + assertEquals(new HashSet<>(oracle), new HashSet<>(cursor)); + } + + private static boolean matchesPattern(Statement s, StatementPattern query) { + return (query.getSubject() == null || query.getSubject().equals(s.getSubject())) + && (query.getPredicate() == null || query.getPredicate().equals(s.getPredicate())) + && (query.getObject() == null || query.getObject().equals(s.getObject())) + && (query.getContexts().length == 0 || java.util.Arrays.asList(query.getContexts()).contains(s.getContext())); + } + + @Test void cursorsCaptureAtFindAndSurviveConcurrentMutationClearAndShutdown() { + var root = storage(); + try { + List input = new ArrayList<>(); + for (int i = 0; i < 40; i++) input.add(row(0, 0, i, null)); + root.mutations().addAll(input); List original = all(root); + try (var unopened = root.queries().find(StatementPattern.matchAll()); + var partial = root.queries().find(StatementPattern.matchAll())) { + var iterator = partial.iterator(); assertSame(original.getFirst(), iterator.next()); + CompletableFuture.runAsync(() -> { + root.mutations().remove(input.get(5)); root.mutations().add(row(2, 3, 4, null)); + root.mutations().clear(); root.close(); + }).join(); + assertEquals(original, unopened.toList()); + List remaining = new ArrayList<>(); iterator.forEachRemaining(remaining::add); + assertEquals(original.subList(1, original.size()), remaining); + for (int i = 0; i < remaining.size(); i++) assertSame(original.get(i + 1), remaining.get(i)); + } + } finally { root.close(); } + } + + @Test void closeExhaustionAndConsumerFailureReleaseExactlyOnce() throws Exception { + try (var root = storage()) { + root.mutations().add(row(0, 0, 0, null)); Object partition = partition(root, null); + var a = root.queries().find(StatementPattern.matchAll()); + var b = root.queries().find(StatementPattern.matchAll()); assertEquals(3, references(partition)); + a.close(); a.close(); assertEquals(2, references(partition)); + b.close(); assertEquals(1, references(partition)); + try (var stream = root.queries().find(StatementPattern.matchAll())) { assertEquals(1, stream.count()); } + assertEquals(1, references(partition)); + assertEquals(1, root.queries().find(StatementPattern.matchAll()).parallel().mapToLong(s -> 1).sum()); + assertEquals(1, references(partition)); + try (var stream = root.queries().find(StatementPattern.matchAll())) { + assertThrows(IllegalArgumentException.class, () -> stream.forEach(s -> { throw new IllegalArgumentException("consumer"); })); + assertEquals(1, references(partition)); + } + try (var stream = root.queries().find(StatementPattern.matchAll())) { + assertThrows(IllegalArgumentException.class, () -> stream.limit(10).forEach(s -> { throw new IllegalArgumentException("partial consumer"); })); + assertEquals(1, references(partition)); + } + try (var stream = root.queries().find(StatementPattern.of(new SimpleIRI("urn:absent"), null, null))) { + assertTrue(stream.findFirst().isEmpty()); assertEquals(1, references(partition)); + } + } + } + + @Test void partialStreamsCloseWithoutKeepingOldPartitionBuffers() throws Exception { + try (var root = storage()) { + for (int i = 0; i < 20; i++) root.mutations().add(row(0, 0, i, null)); + Object original = partition(root, null); + var stream = root.queries().find(StatementPattern.matchAll()); + var cursor = stream.spliterator(); assertTrue(cursor.tryAdvance(s -> {})); + root.mutations().add(row(1, 1, 1, null)); assertNotSame(original, partition(root, null)); + assertEquals(1, references(original)); stream.close(); assertEquals(0, references(original)); + assertFalse(cursor.tryAdvance(s -> fail("closed cursor"))); + Object writer = partition(root, null); root.mutations().add(row(1, 1, 2, null)); + assertSame(writer, partition(root, null)); + } + } + + @Test void explicitContextsDoNotShareUnselectedGraphsAndKeepRegistryOrder() throws Exception { + try (var root = storage()) { + root.mutations().addAll(List.of(row(0, 0, 0, G), row(0, 0, 1, H), row(0, 0, 2, null))); + Object unselected = partition(root, H); + try (var cursor = root.queries().find(StatementPattern.of(null, null, null, G, G))) { + root.mutations().add(row(0, 0, 3, H)); assertSame(unselected, partition(root, H)); + assertEquals(List.of(row(0, 0, 0, G)), cursor.toList()); + } + try (var cursor = root.queries().find(StatementPattern.of(null, null, null, null, G))) { + assertEquals(List.of(row(0, 0, 0, G), row(0, 0, 2, null)), cursor.toList()); + } + } + } + + @Test void cursorOwnsItsVersionAcrossTransactionCommitAndRollback() { + try (var root = storage(); var session = root.openSession()) { + root.mutations().add(row(0, 0, 0, null)); + try (var transaction = session.transactions().beginTransaction(); + var cursor = session.queries().find(StatementPattern.matchAll())) { + session.mutations().add(row(0, 0, 1, null)); transaction.commit(); + assertEquals(List.of(row(0, 0, 0, null)), cursor.toList()); assertEquals(2, all(root).size()); + } + try (var transaction = session.transactions().beginTransaction()) { + session.mutations().add(row(0, 0, 2, null)); + try (var cursor = session.queries().find(StatementPattern.matchAll())) { + transaction.rollback(); assertEquals(3, cursor.toList().size()); assertEquals(2, all(root).size()); + } + } + } + } + + @Test void threeIndependentReadersAndWriterCanProgressConcurrently() { + try (var root = storage()) { + List input = new ArrayList<>(); + for (int s = 0; s < 120; s++) { + for (int p = 0; p < 12; p++) { + for (int o = 0; o < 12; o++) { + input.add(row(s, p, o, null)); + } + } + } + root.mutations().addAll(input); + List expected = all(root); + var start = new CountDownLatch(1); + List> tasks = new ArrayList<>(); + for (int r = 0; r < 3; r++) { + tasks.add(launchConcurrentReader(root, start, expected)); + } + tasks.add(launchConcurrentWriter(root, start, input)); + start.countDown(); + CompletableFuture.allOf(tasks.toArray(CompletableFuture[]::new)).join(); + MemoryStorageAssertions.verify((IndexedStatementStore) root.queries()); + } + } + + private static CompletableFuture launchConcurrentReader(StorageManager root, + CountDownLatch start, List expected) { + var stream = root.queries().find(StatementPattern.matchAll()); + return CompletableFuture.runAsync(() -> { + try (stream) { + try { + start.await(); + } catch (InterruptedException interrupted) { + Thread.currentThread().interrupt(); + throw new AssertionError(interrupted); + } + List actual = new ArrayList<>(); + var iterator = stream.iterator(); + while (iterator.hasNext()) { + actual.add(iterator.next()); + if (actual.size() % 100 == 0) { + Thread.yield(); + } + } + assertEquals(expected, actual); + } + }); + } + + private static CompletableFuture launchConcurrentWriter(StorageManager root, + CountDownLatch start, List input) { + return CompletableFuture.runAsync(() -> { + try { + start.await(); + } catch (InterruptedException interrupted) { + Thread.currentThread().interrupt(); + throw new AssertionError(interrupted); + } + for (int i = 0; i < 50; i++) { + root.mutations().remove(input.get(i)); + root.mutations().add(row(500, 0, i, null)); + } + }); + } +} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryDeletionTest.java b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryDeletionTest.java new file mode 100644 index 000000000..0663e2919 --- /dev/null +++ b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryDeletionTest.java @@ -0,0 +1,302 @@ +package fr.inria.corese.core.next.storage.impl.memory; + +import fr.inria.corese.core.next.data.Values; +import fr.inria.corese.core.next.data.api.model.Statement; +import fr.inria.corese.core.next.data.api.term.Resource; +import fr.inria.corese.core.next.data.api.term.IRI; +import fr.inria.corese.core.next.data.api.term.Value; +import fr.inria.corese.core.next.data.api.term.SimpleIRI; +import fr.inria.corese.core.next.storage.Storages; +import fr.inria.corese.core.next.storage.api.StorageManager; +import fr.inria.corese.core.next.storage.api.config.StorageConfig; +import fr.inria.corese.core.next.storage.api.exception.StorageException; +import fr.inria.corese.core.next.storage.api.model.StatementPattern; +import org.junit.jupiter.api.Test; +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Set; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.TimeoutException; +import java.util.concurrent.TimeUnit; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertSame; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; + +class MemoryDeletionTest { + private static final StorageConfig CONFIG = StorageConfig.memory(); + private static final SimpleIRI S = new SimpleIRI("urn:delete:s"); + private static final SimpleIRI P = new SimpleIRI("urn:delete:p"); + private static final SimpleIRI G = new SimpleIRI("urn:delete:g"); + + private static Statement statement(int i, Resource graph) { + return Values.factory().createStatement(S, P, new SimpleIRI("urn:delete:o" + i), graph); + } + + private static List rows(StorageManager storage) { + try (var stream = storage.queries().find(StatementPattern.matchAll())) { + return stream.toList(); + } + } + + private static void verify(StorageManager storage) { + MemoryStorageAssertions.verify((IndexedStatementStore) storage.queries()); + } + + @Test + void clearRetainsGraphOrderCanonicalContextsAndThreeConcurrentSnapshots() + throws InterruptedException, ExecutionException, TimeoutException { + try (var root = Storages.create(CONFIG)) { + var later = new SimpleIRI("urn:delete:later"); + root.mutations().add(statement(0, G)); + root.mutations().add(statement(1, null)); + root.mutations().add(statement(2, later)); + List original = rows(root); + var ready = new CountDownLatch(3); + var cleared = new CountDownLatch(1); + List> tasks = new ArrayList<>(); + for (int i = 0; i < 3; i++) { + tasks.add(startSnapshotReader(root, original, ready, cleared)); + } + try { + assertTrue(ready.await(10, TimeUnit.SECONDS)); + // Duplicate selectors, reversed order, default and absent graph. + assertEquals(2, root.mutations().clear(null, G, G, new SimpleIRI("urn:absent"))); + assertEquals(Set.of(G, later), root.metadata().getContexts()); + assertEquals(0, root.mutations().clear(null, G)); + // Reinsert in reverse order: graph registry order must still win. + root.mutations().add(statement(3, null)); + root.mutations().add(statement(4, new SimpleIRI(G.stringValue()))); + List current = rows(root); + assertEquals(List.of(statement(4, G), statement(3, null), statement(2, later)), current); + assertSame(original.getFirst().getContext(), current.getFirst().getContext()); + verify(root); + } finally { + cleared.countDown(); + } + CompletableFuture.allOf(tasks.toArray(CompletableFuture[]::new)).get(10, TimeUnit.SECONDS); + assertEquals(3, root.mutations().remove(StatementPattern.matchAll())); + assertEquals(Set.of(G, later), root.metadata().getContexts()); + assertTrue(root.mutations().dropGraph(G)); + assertEquals(Set.of(later), root.metadata().getContexts()); + verify(root); + } + } + + private static CompletableFuture startSnapshotReader(StorageManager root, List original, + CountDownLatch ready, CountDownLatch cleared) { + return CompletableFuture.runAsync(() -> { + try (var reader = root.openSession(); var tx = reader.transactions().beginTransaction()) { + ready.countDown(); + assertTrue(cleared.await(10, TimeUnit.SECONDS)); + assertSameStatements(original, rows(reader)); + verify(reader); + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + throw new AssertionError(e); + } + }); + } + + private static void assertSameStatements(List expected, List actual) { + assertEquals(expected, actual); + for (int i = 0; i < expected.size(); i++) { + assertSame(expected.get(i), actual.get(i)); + } + } + + @Test + void clearRollbackCommitConflictsAndNoOpVersionsPreserveOwners() { + try (var root = Storages.create(CONFIG); var writer = root.openSession(); var reader = root.openSession()) { + var input = List.of(statement(0, G), statement(1, null)); + root.mutations().addAll(input); + try (var snapshot = reader.transactions().beginTransaction()) { + try (var tx = writer.transactions().beginTransaction()) { + assertEquals(2, writer.mutations().clear()); + assertEquals(input, rows(root)); + assertEquals(input, rows(reader)); + tx.rollback(); + } + assertEquals(input, rows(writer)); + try (var tx = writer.transactions().beginTransaction()) { + assertEquals(2, writer.mutations().clear()); + writer.mutations().add(statement(2, G)); + tx.commit(); + } + assertEquals(input, rows(reader)); + assertEquals(List.of(statement(2, G)), rows(root)); + assertThrows(StorageException.class, snapshot::commit); + verify(reader); + verify(root); + } + try (var tx = writer.transactions().beginTransaction()) { + assertEquals(0, root.mutations().clear((Resource) null)); + assertEquals(0, root.mutations().clear(new SimpleIRI("urn:absent"))); + writer.mutations().add(statement(3, G)); + tx.commit(); // Empty clears must not create conflicts or missing graphs. + } + assertFalse(root.metadata().getContexts().contains(new SimpleIRI("urn:absent"))); + verify(root); + } + } + + @Test + void batchFilteringMatchesOrderedOracleForAllMasksAndPermutations() { + List input = createIndexStatements(); + for (TripleIndex.Order order : TripleIndex.Order.values()) { + for (int mask = 0; mask < 8; mask++) { + assertOrderedFiltering(input, order, mask); + } + } + } + + private static List createIndexStatements() { + List input = new ArrayList<>(); + for (int s = 0; s < 16; s++) { + for (int p = 0; p < 4; p++) { + for (int o = 0; o < 16; o++) { + input.add(Values.factory().createStatement(new SimpleIRI("urn:s" + s), new SimpleIRI("urn:p" + p), + new SimpleIRI("urn:o" + o))); + } + } + } + return input; + } + + private static void assertOrderedFiltering(List input, TripleIndex.Order order, int mask) { + TripleIndex index = new TripleIndex(order); + for (Statement s : input) { + MemoryStorageAssertions.add(index, s); + } + index.finishStaging(); + TripleIndex snapshot = index.copy(); + var subject = (mask & 4) == 0 ? null : input.getFirst().getSubject(); + var predicate = (mask & 2) == 0 ? null : input.getFirst().getPredicate(); + var object = (mask & 1) == 0 ? null : input.getFirst().getObject(); + List survivors = input.stream().filter(s -> !matches(s, subject, predicate, object)).toList(); + List before = new ArrayList<>(); + snapshot.visit(StatementPattern.matchAll(), s -> { + before.add(s); + return false; + }); + assertEquals((long) input.size() - survivors.size(), + index.removeMatches(TermKey.nullable(subject), TermKey.nullable(predicate), TermKey.nullable(object))); + List actual = new ArrayList<>(); + index.visit(StatementPattern.matchAll(), s -> { + actual.add(s); + return false; + }); + assertEquals(before.stream().filter(survivors::contains).toList(), actual); + assertEquals(survivors.size(), index.size()); + assertEquals(0, + index.removeMatches(TermKey.nullable(subject), TermKey.nullable(predicate), TermKey.nullable(object))); + List unchanged = new ArrayList<>(); + snapshot.visit(StatementPattern.matchAll(), s -> { + unchanged.add(s); + return false; + }); + assertEquals(before, unchanged); + restoreRemovedStatements(index, input, survivors); + index.finishStaging(); + assertEquals(input.size(), index.size()); + for (Statement s : input) { + assertEquals(1, index.count(StatementPattern.of(s.getSubject(), s.getPredicate(), s.getObject()))); + } + } + + private static boolean matches(Statement statement, Resource subject, IRI predicate, Value object) { + return (subject == null || subject.equals(statement.getSubject())) + && (predicate == null || predicate.equals(statement.getPredicate())) + && (object == null || object.equals(statement.getObject())); + } + + private static void restoreRemovedStatements(TripleIndex index, List input, List survivors) { + for (Statement statement : input) { + if (!survivors.contains(statement)) { + assertTrue(MemoryStorageAssertions.add(index, statement)); + } + } + } + + @Test + void broadPatternsAcrossGraphsRetainTermsAndReaderVersions() { + try (var root = Storages.create(CONFIG); var reader = root.openSession()) { + List input = createGraphStatements(); + root.mutations().addAll(input); + List before = rows(root); + try (var tx = reader.transactions().beginTransaction()) { + assertEquals(512, root.mutations().remove(StatementPattern.of(null, P, null, G, null))); + var actual = rows(root); + var expected = before.stream().filter(s -> !s.getPredicate().equals(P)).toList(); + assertSameStatements(expected, actual); + assertEquals(before, rows(reader)); + verify(root); + verify(reader); + // Bound pattern covers an entire remaining partition: same fast clear contract. + assertEquals(768, + root.mutations().remove(StatementPattern.of(null, new SimpleIRI("urn:keep"), null, G))); + assertEquals(Set.of(G), root.metadata().getContexts()); + assertEquals(before, rows(reader)); + root.mutations().addAll(input); + verify(root); + verify(reader); + } + } + } + + private static List createGraphStatements() { + List input = new ArrayList<>(); + for (Resource graph : new Resource[] { null, G }) { + for (int i = 0; i < 1024; i++) { + input.add(Values.factory().createStatement(new SimpleIRI("urn:s" + (i / 8)), + i % 4 == 0 ? P : new SimpleIRI("urn:keep"), new SimpleIRI("urn:o" + i), graph)); + } + } + return input; + } + + @Test + void denseBatchFilteringHandlesHolesCollisionsNoOpsAndReinsertion() { + for (boolean collision : List.of(false, true)) { + assertDenseBatchFiltering(collision); + } + } + + private static void assertDenseBatchFiltering(boolean collision) { + var table = new DenseTermTable(); + var oracle = new LinkedHashMap(); + for (int i = 0; i < 512; i++) { + var key = DenseTermTableTest.key(i, collision); + table.put(key, i); + oracle.put(key, i); + } + for (int i = 0; i < 50; i++) { + var key = DenseTermTableTest.key(i, collision); + table.remove(key); + oracle.remove(key); + } + var copy = new DenseTermTable<>(table); + assertEquals(0, table.removeIf(i -> false)); + for (var entry : oracle.entrySet()) { + assertEquals(entry.getValue(), table.get(entry.getKey())); + } + int size = oracle.size(); + oracle.values().removeIf(i -> i % 3 != 0); + assertEquals(size - oracle.size(), table.removeIf(i -> i % 3 != 0)); + assertEquals(new ArrayList<>(oracle.values()), new ArrayList<>(table.values())); + var keys = table.keySet().iterator(); + for (var entry : oracle.entrySet()) { + assertSame(entry.getKey(), keys.next()); + assertEquals(entry.getValue(), table.get(entry.getKey())); + } + assertEquals(size, copy.size()); + assertEquals(table.size(), table.removeIf(i -> true)); + assertEquals(0, table.removeIf(i -> true)); + table.put(DenseTermTableTest.key(0, collision), 7); + assertEquals(7, table.get(DenseTermTableTest.key(0, collision))); + } +} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryLifecycleManagerTest.java b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryLifecycleManagerTest.java deleted file mode 100644 index 9d6f35377..000000000 --- a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryLifecycleManagerTest.java +++ /dev/null @@ -1,119 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.memory; - -import fr.inria.corese.core.next.storage.api.lifecycle.LifecycleState; -import fr.inria.corese.core.next.storage.api.config.StorageConfig; -import fr.inria.corese.core.next.storage.api.exception.StorageException; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.DisplayName; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; - -import fr.inria.corese.core.next.data.api.model.Statement; -import static org.mockito.Mockito.mock; - -import static org.junit.jupiter.api.Assertions.*; - -@DisplayName("MemoryLifecycleManager tests") -class MemoryLifecycleManagerTest { - - private InMemoryStatementStore adapter; - private MemoryLifecycleManager lifecycleManager; - - @BeforeEach - void setUp() { - adapter = new InMemoryStatementStore(); - lifecycleManager = new MemoryLifecycleManager(adapter); - } - - @Nested - @DisplayName("Constructor tests") - class ConstructorTests { - - @Test - @DisplayName("Should throw when adapter is null") - void shouldThrowWhenAdapterIsNull() { - assertThrows(IllegalArgumentException.class, - () -> new MemoryLifecycleManager(null)); - } - - @Test - @DisplayName("Should start in NOT_INITIALIZED state") - void shouldStartInNotInitializedState() { - assertEquals(LifecycleState.NOT_INITIALIZED, - lifecycleManager.getState()); - } - } - - @Nested - @DisplayName("initialize() tests") - class InitializeTests { - - @Test - @DisplayName("Should throw when config is null") - void shouldThrowWhenConfigIsNull() { - assertThrows(IllegalArgumentException.class, - () -> lifecycleManager.initialize(null)); - } - - @Test - @DisplayName("Should initialize successfully") - void shouldInitializeSuccessfully() throws StorageException { - StorageConfig config = StorageConfig.builder().build(); - - lifecycleManager.initialize(config); - - assertEquals(LifecycleState.RUNNING, lifecycleManager.getState()); - assertEquals(java.util.Optional.of(config), lifecycleManager.getConfig()); - } - - @Test - @DisplayName("Should throw when already initialized") - void shouldThrowWhenAlreadyInitialized() throws StorageException { - StorageConfig config = StorageConfig.builder().build(); - lifecycleManager.initialize(config); - - assertThrows(IllegalStateException.class, - () -> lifecycleManager.initialize(config)); - } - } - - @Nested - @DisplayName("shutdown() tests") - class ShutdownTests { - - @Test - @DisplayName("Should throw when not running") - void shouldThrowWhenNotRunning() { - assertThrows(IllegalStateException.class, - () -> lifecycleManager.shutdown()); - } - - @Test - @DisplayName("Should shutdown successfully") - void shouldShutdownSuccessfully() throws StorageException { - StorageConfig config = StorageConfig.builder().build(); - lifecycleManager.initialize(config); - - lifecycleManager.shutdown(); - - assertEquals(LifecycleState.SHUTDOWN, lifecycleManager.getState()); - assertTrue(lifecycleManager.getConfig().isEmpty()); - } - - @Test - @DisplayName("Should not clear data on shutdown") - void shouldNotClearDataOnShutdown() throws StorageException { - // Add some data - adapter.add(mock(Statement.class)); - int sizeBefore = adapter.size(); - - // Initialize and shutdown - StorageConfig config = StorageConfig.builder().build(); - lifecycleManager.initialize(config); - lifecycleManager.shutdown(); - - // Data should still be there - assertEquals(sizeBefore, adapter.size()); - } - } -} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryMetadataOperationsTest.java b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryMetadataOperationsTest.java deleted file mode 100644 index 3dcce7d3a..000000000 --- a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryMetadataOperationsTest.java +++ /dev/null @@ -1,286 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.memory; - -import fr.inria.corese.core.next.data.api.term.IRI; -import fr.inria.corese.core.next.data.api.term.Resource; -import fr.inria.corese.core.next.data.api.term.Value; -import fr.inria.corese.core.next.storage.api.exception.StorageException; -import fr.inria.corese.core.next.storage.api.model.StorageStatistics; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.DisplayName; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; -import org.mockito.Mock; -import org.mockito.MockitoAnnotations; - -import java.util.Arrays; -import java.util.Collections; -import java.util.HashSet; -import java.util.Set; - -import static org.junit.jupiter.api.Assertions.*; -import static org.mockito.Mockito.*; - -/** - * Unit tests for MemoryMetadataOperations. - */ -class MemoryMetadataOperationsTest { - - @Mock - private InMemoryStatementStore mockAdapter; - - private MemoryMetadataOperations metadataOps; - - @BeforeEach - void setUp() { - MockitoAnnotations.openMocks(this); - metadataOps = new MemoryMetadataOperations(mockAdapter); - } - - @Nested - @DisplayName("Constructor tests") - class ConstructorTests { - - @Test - @DisplayName("Should throw when adapter is null") - void shouldThrowWhenAdapterIsNull() { - assertThrows(IllegalArgumentException.class, - () -> new MemoryMetadataOperations(null)); - } - } - - @Nested - @DisplayName("getPredicates() tests") - class GetPredicatesTests { - - @Test - @DisplayName("Should return unmodifiable set of predicates") - void shouldReturnUnmodifiableSetOfPredicates() throws StorageException { - IRI predicate1 = mock(IRI.class); - IRI predicate2 = mock(IRI.class); - Set mockPredicates = new HashSet<>(Arrays.asList(predicate1, predicate2)); - - when(mockAdapter.getPredicates()).thenReturn(mockPredicates); - - Set result = metadataOps.getPredicates(); - - assertEquals(2, result.size()); - assertTrue(result.contains(predicate1)); - assertTrue(result.contains(predicate2)); - verify(mockAdapter).getPredicates(); - } - - @Test - @DisplayName("Should return unmodifiable set") - void shouldReturnUnmodifiableSet() throws StorageException { - when(mockAdapter.getPredicates()).thenReturn(new HashSet<>()); - - Set result = metadataOps.getPredicates(); - - assertThrows(UnsupportedOperationException.class, result::clear); - } - - @Test - @DisplayName("Should return empty set when no predicates") - void shouldReturnEmptySetWhenNoPredicates() throws StorageException { - when(mockAdapter.getPredicates()).thenReturn(Collections.emptySet()); - - Set result = metadataOps.getPredicates(); - - assertTrue(result.isEmpty()); - } - } - - @Nested - @DisplayName("getSubjects() tests") - class GetSubjectsTests { - - @Test - @DisplayName("Should return unmodifiable set of subjects") - void shouldReturnUnmodifiableSetOfSubjects() { - Resource subject1 = mock(Resource.class); - Resource subject2 = mock(Resource.class); - Set mockSubjects = new HashSet<>(Arrays.asList(subject1, subject2)); - - when(mockAdapter.getSubjects()).thenReturn(mockSubjects); - - Set result = metadataOps.getSubjects(); - - assertEquals(2, result.size()); - assertTrue(result.contains(subject1)); - assertTrue(result.contains(subject2)); - verify(mockAdapter).getSubjects(); - } - - @Test - @DisplayName("Should return unmodifiable set") - void shouldReturnUnmodifiableSet() { - when(mockAdapter.getSubjects()).thenReturn(new HashSet<>()); - - Set result = metadataOps.getSubjects(); - - assertThrows(UnsupportedOperationException.class, result::clear); - } - } - - @Nested - @DisplayName("getObjects() tests") - class GetObjectsTests { - - @Test - @DisplayName("Should return unmodifiable set of objects") - void shouldReturnUnmodifiableSetOfObjects() { - Value object1 = mock(Value.class); - Value object2 = mock(Value.class); - Value object3 = mock(Value.class); - Set mockObjects = new HashSet<>(Arrays.asList(object1, object2, object3)); - - when(mockAdapter.getObjects()).thenReturn(mockObjects); - - Set result = metadataOps.getObjects(); - - assertEquals(3, result.size()); - assertTrue(result.contains(object1)); - assertTrue(result.contains(object2)); - assertTrue(result.contains(object3)); - verify(mockAdapter).getObjects(); - } - - @Test - @DisplayName("Should return unmodifiable set") - void shouldReturnUnmodifiableSet() { - when(mockAdapter.getObjects()).thenReturn(new HashSet<>()); - - Set result = metadataOps.getObjects(); - - assertThrows(UnsupportedOperationException.class, result::clear); - } - } - - @Nested - @DisplayName("getContexts() tests") - class GetContextsTests { - - @Test - @DisplayName("Should return unmodifiable set of contexts") - void shouldReturnUnmodifiableSetOfContexts() throws StorageException { - Resource context1 = mock(Resource.class); - Resource context2 = mock(Resource.class); - Set mockContexts = new HashSet<>(Arrays.asList(context1, context2)); - - when(mockAdapter.getContexts()).thenReturn(mockContexts); - - Set result = metadataOps.getContexts(); - - assertEquals(2, result.size()); - assertTrue(result.contains(context1)); - assertTrue(result.contains(context2)); - verify(mockAdapter).getContexts(); - } - - @Test - @DisplayName("Should return unmodifiable set") - void shouldReturnUnmodifiableSet() throws StorageException { - when(mockAdapter.getContexts()).thenReturn(new HashSet<>()); - - Set result = metadataOps.getContexts(); - - assertThrows(UnsupportedOperationException.class, result::clear); - } - - @Test - @DisplayName("Should return empty set when no contexts") - void shouldReturnEmptySetWhenNoContexts() throws StorageException { - when(mockAdapter.getContexts()).thenReturn(Collections.emptySet()); - - Set result = metadataOps.getContexts(); - - assertTrue(result.isEmpty()); - } - } - - @Nested - @DisplayName("getStatistics() tests") - class GetStatisticsTests { - - @Test - @DisplayName("Should return statistics with correct counts") - void shouldReturnStatisticsWithCorrectCounts() { - // Setup mocks - when(mockAdapter.size()).thenReturn(100); - - Set subjects = new HashSet<>(Arrays.asList( - mock(Resource.class), mock(Resource.class), mock(Resource.class) - )); - when(mockAdapter.getSubjects()).thenReturn(subjects); - - Set predicates = new HashSet<>(Arrays.asList( - mock(IRI.class), mock(IRI.class) - )); - when(mockAdapter.getPredicates()).thenReturn(predicates); - - Set objects = new HashSet<>(Arrays.asList( - mock(Value.class), mock(Value.class), mock(Value.class), mock(Value.class) - )); - when(mockAdapter.getObjects()).thenReturn(objects); - - Set contexts = new HashSet<>(Collections.singletonList( - mock(Resource.class) - )); - when(mockAdapter.getContexts()).thenReturn(contexts); - - // Get statistics - StorageStatistics stats = metadataOps.getStatistics(); - - // Verify - assertEquals(100, stats.statementCount()); - assertEquals(3, stats.subjectCount()); - assertEquals(2, stats.predicateCount()); - assertEquals(4, stats.objectCount()); - assertEquals(1, stats.contextCount()); - - verify(mockAdapter).size(); - verify(mockAdapter).getSubjects(); - verify(mockAdapter).getPredicates(); - verify(mockAdapter).getObjects(); - verify(mockAdapter).getContexts(); - } - - @Test - @DisplayName("Should return zero statistics for empty graph") - void shouldReturnZeroStatisticsForEmptyGraph() { - when(mockAdapter.size()).thenReturn(0); - when(mockAdapter.getSubjects()).thenReturn(Collections.emptySet()); - when(mockAdapter.getPredicates()).thenReturn(Collections.emptySet()); - when(mockAdapter.getObjects()).thenReturn(Collections.emptySet()); - when(mockAdapter.getContexts()).thenReturn(Collections.emptySet()); - - StorageStatistics stats = metadataOps.getStatistics(); - - assertEquals(0, stats.statementCount()); - assertEquals(0, stats.subjectCount()); - assertEquals(0, stats.predicateCount()); - assertEquals(0, stats.objectCount()); - assertEquals(0, stats.contextCount()); - } - - @Test - @DisplayName("Should call adapter methods for statistics") - void shouldCallAdapterMethodsForStatistics() { - // Setup with empty sets to simplify - when(mockAdapter.size()).thenReturn(0); - when(mockAdapter.getSubjects()).thenReturn(Collections.emptySet()); - when(mockAdapter.getPredicates()).thenReturn(Collections.emptySet()); - when(mockAdapter.getObjects()).thenReturn(Collections.emptySet()); - when(mockAdapter.getContexts()).thenReturn(Collections.emptySet()); - - metadataOps.getStatistics(); - - // Verify all methods were called - verify(mockAdapter, times(1)).size(); - verify(mockAdapter, times(1)).getSubjects(); - verify(mockAdapter, times(1)).getPredicates(); - verify(mockAdapter, times(1)).getObjects(); - verify(mockAdapter, times(1)).getContexts(); - } - } -} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryMutationOperationsTest.java b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryMutationOperationsTest.java deleted file mode 100644 index cb082e22c..000000000 --- a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryMutationOperationsTest.java +++ /dev/null @@ -1,83 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.memory; - -import fr.inria.corese.core.next.data.api.model.Statement; -import fr.inria.corese.core.next.data.api.term.Resource; -import fr.inria.corese.core.next.storage.api.exception.StorageException; -import fr.inria.corese.core.next.storage.api.model.StatementPattern; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.Test; - -import java.util.Set; - -import static org.junit.jupiter.api.Assertions.*; -import static org.mockito.ArgumentMatchers.*; -import static org.mockito.Mockito.*; - -class MemoryMutationOperationsTest { - - private InMemoryStatementStore store; - private MemoryMutationOperations mutations; - - @BeforeEach - void setUp() { - store = mock(InMemoryStatementStore.class); - mutations = new MemoryMutationOperations(store); - } - - @Test - void rejectsMissingDependenciesAndArguments() { - assertThrows(IllegalArgumentException.class, () -> new MemoryMutationOperations(null)); - assertThrows(NullPointerException.class, () -> mutations.add(null)); - assertThrows(NullPointerException.class, () -> mutations.remove((Statement) null)); - assertThrows(NullPointerException.class, () -> mutations.remove((StatementPattern) null)); - } - - @Test - void reportsWhetherSingleStatementMutationsChangedTheStore() { - Statement statement = mock(Statement.class); - when(store.add(statement)).thenReturn(true, false); - when(store.remove(statement)).thenReturn(true, false); - - assertTrue(mutations.add(statement)); - assertFalse(mutations.add(statement)); - assertTrue(mutations.remove(statement)); - assertFalse(mutations.remove(statement)); - } - - @Test - void wrapsBackendFailures() { - Statement statement = mock(Statement.class); - when(store.add(statement)).thenThrow(new IllegalStateException("failed")); - - assertThrows(StorageException.class, () -> mutations.add(statement)); - } - - @Test - void removesMatchingStatementsAndReturnsTheirCount() { - Statement first = mock(Statement.class); - Statement second = mock(Statement.class); - when(store.find(isNull(), isNull(), isNull(), any(Resource[].class))) - .thenReturn(Set.of(first, second)); - - assertEquals(2, mutations.remove(StatementPattern.matchAll())); - verify(store).remove(first); - verify(store).remove(second); - } - - @Test - void clearsAllOrSelectedGraphsAndReturnsTheRemovedCount() { - Resource first = mock(Resource.class); - Resource second = mock(Resource.class); - Statement firstMatch = mock(Statement.class); - Statement secondMatch = mock(Statement.class); - when(store.size()).thenReturn(10, 0); - when(store.find(null, null, null, new Resource[]{first, second})) - .thenReturn(Set.of(firstMatch, secondMatch)); - - assertEquals(10, mutations.clear()); - assertEquals(2, mutations.clear(first, second)); - verify(store).clear(); - verify(store).remove(firstMatch); - verify(store).remove(secondMatch); - } -} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryQueryOperationsTest.java b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryQueryOperationsTest.java deleted file mode 100644 index becf370be..000000000 --- a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryQueryOperationsTest.java +++ /dev/null @@ -1,118 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.memory; - -import fr.inria.corese.core.next.data.api.model.Statement; -import fr.inria.corese.core.next.storage.api.exception.StorageException; -import fr.inria.corese.core.next.storage.api.model.StatementPattern; -import org.junit.jupiter.api.*; -import org.mockito.Mock; -import org.mockito.MockitoAnnotations; - -import java.util.*; -import java.util.stream.Stream; - -import static org.junit.jupiter.api.Assertions.*; -import static org.mockito.Mockito.*; - -/** - * Unit tests for MemoryQueryOperations. - */ -class MemoryQueryOperationsTest { - - @Mock - private InMemoryStatementStore mockAdapter; - - private MemoryQueryOperations queryOps; - - @BeforeEach - void setUp() { - MockitoAnnotations.openMocks(this); - queryOps = new MemoryQueryOperations(mockAdapter); - } - - @Nested - @DisplayName("Constructor tests") - class ConstructorTests { - - @Test - @DisplayName("Should throw when adapter is null") - void shouldThrowWhenAdapterIsNull() { - assertThrows(NullPointerException.class, - () -> new MemoryQueryOperations(null)); - } - } - - @Nested - @DisplayName("query() tests") - class QueryTests { - - @Test - @DisplayName("Should throw when pattern is null") - void shouldThrowWhenPatternIsNull() { - assertThrows(NullPointerException.class, - () -> queryOps.find(null)); - } - - @Test - @DisplayName("Should return stream of statements") - void shouldReturnStreamOfStatements() throws StorageException { - Statement mockStatement = mock(Statement.class); - StatementPattern pattern = StatementPattern.of(null, null, null); - Set mockResults = new HashSet<>(Collections.singletonList(mockStatement)); - - when(mockAdapter.find(null, null, null, null)) - .thenAnswer(invocation -> new HashSet<>(mockResults)); - - Stream result = queryOps.find(pattern); - - assertNotNull(result); - } - - @Test - @DisplayName("Should wrap exceptions in StorageException") - void shouldWrapExceptionsInStorageException() { - StatementPattern pattern = StatementPattern.of(null, null, null); - when(mockAdapter.find(any(), any(), any(), any())) - .thenThrow(new RuntimeException("Test error")); - - assertThrows(StorageException.class, - () -> queryOps.find(pattern)); - } - } - - @Nested - @DisplayName("count() tests") - class CountTests { - - - @Test - @DisplayName("Should return 0 for empty results") - void shouldReturnZeroForEmptyResults() throws StorageException { - StatementPattern pattern = StatementPattern.of(null, null, null); - when(mockAdapter.find(null, null, null, null)) - .thenReturn(Collections.emptySet()); - - long count = queryOps.count(pattern); - - assertEquals(0, count); - } - } - - @Nested - @DisplayName("exists() tests") - class ExistsTests { - - - @Test - @DisplayName("Should return false when no statements exist") - void shouldReturnFalseWhenNoStatementsExist() throws StorageException { - StatementPattern pattern = StatementPattern.of(null, null, null); - when(mockAdapter.find(null, null, null, null)) - .thenReturn(Collections.emptySet()); - - boolean exists = queryOps.contains(pattern); - - assertFalse(exists); - } - } - -} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemorySizeAgent.java b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemorySizeAgent.java new file mode 100644 index 000000000..a7cfaee38 --- /dev/null +++ b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemorySizeAgent.java @@ -0,0 +1,23 @@ +package fr.inria.corese.core.next.storage.impl.memory; + +import java.lang.instrument.Instrumentation; + +/** Standalone profiler agent; not loaded by production or JUnit. */ +public final class MemorySizeAgent { + private static Instrumentation instrumentation; + + private MemorySizeAgent() { + // Utility / agent class + } + + // The JVM agent entry point requires this two-parameter signature, even without agent options. + @SuppressWarnings("java:S1172") + public static void premain(String arguments, Instrumentation agent) { + instrumentation = agent; + } + + static long size(Object value) { + if (instrumentation == null) throw new IllegalStateException("Start with the profiling -javaagent jar"); + return instrumentation.getObjectSize(value); + } +} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryStorageAssertions.java b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryStorageAssertions.java new file mode 100644 index 000000000..1b89a98c9 --- /dev/null +++ b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryStorageAssertions.java @@ -0,0 +1,119 @@ +package fr.inria.corese.core.next.storage.impl.memory; + +import fr.inria.corese.core.next.data.api.model.Statement; +import fr.inria.corese.core.next.storage.api.model.StatementPattern; + +import java.lang.reflect.Field; +import java.lang.reflect.InvocationTargetException; +import java.lang.reflect.Method; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Objects; +import java.util.Set; + +/** + * Structural checks shared by storage tests and benchmark correctness oracles. + * Reflection keeps diagnostic access out of production classes. These checks + * run under the database monitor and must remain outside benchmark timers. + */ +public final class MemoryStorageAssertions { + private MemoryStorageAssertions() { } + + static boolean add(TripleIndex index, Statement statement) { + return index.add(statement, statement.getContext()); + } + + static boolean remove(TripleIndex index, Statement statement) { + return index.remove(new TermKey(statement.getSubject()), new TermKey(statement.getPredicate()), + new TermKey(statement.getObject())); + } + + /** Checks counts, contexts, permutation parity and canonical key identity. */ + public static void verify(IndexedStatementStore store) { + synchronized (field(store, "database")) { + invoke(store, "checkAccess"); + Object state = invoke(store, "state"); + Map graphs = field(state, "graphs"); + long total = 0; + for (var entry : graphs.entrySet()) { + Object partition = entry.getValue(); + TripleIndex spo = field(partition, "spo"); + TripleIndex pos = field(partition, "pos"); + TripleIndex osp = field(partition, "osp"); + verifyTerms(spo, pos, osp); + Set subjects = statements(spo); + Set predicates = statements(pos); + Set objects = statements(osp); + if (!subjects.equals(predicates) || !subjects.equals(objects) + || subjects.size() != spo.size() || predicates.size() != pos.size() + || objects.size() != osp.size() + || subjects.stream().anyMatch(s -> !Objects.equals(s.getContext(), entry.getKey()))) { + throw new IllegalStateException("Inconsistent index partition"); + } + total += subjects.size(); + } + if (total != (long) field(state, "size")) { + throw new IllegalStateException("Inconsistent statement count"); + } + } + } + + private static Set statements(TripleIndex index) { + Set result = new HashSet<>(); + index.visit(StatementPattern.matchAll(), statement -> { result.add(statement); return false; }); + return result; + } + + private static void verifyTerms(TripleIndex spo, TripleIndex pos, TripleIndex osp) { + if (field(spo, "stagingTerms") != null) throw new IllegalStateException("Published bulk cache"); + Map canonical = new HashMap<>(); + List indexes = List.of(spo, pos, osp); + for (TripleIndex index : indexes) roots(index).forEach((key, branch) -> + canonical.putIfAbsent(key, field(branch, "key"))); + for (TripleIndex index : indexes) roots(index).forEach((first, branch) -> { + verifyCanonical(canonical, first); + branch.forEach((second, entry) -> { + verifyCanonical(canonical, second); + if (entry instanceof CompactTermMap leaf) { + leaf.forEach((third, statement) -> verifyCanonical(canonical, third)); + } + }); + }); + } + + private static void verifyCanonical(Map canonical, TermKey key) { + if (canonical.get(key) != key) throw new IllegalStateException("Non-canonical term key"); + } + + private static Map> roots(TripleIndex index) { + return field(index, "roots"); + } + + @SuppressWarnings({"java:S3011", "unchecked"}) + private static T field(Object target, String name) { + try { + Field field = target.getClass().getDeclaredField(name); + field.setAccessible(true); + return (T) field.get(target); + } catch (ReflectiveOperationException e) { + throw new AssertionError("Cannot inspect " + name, e); + } + } + + @SuppressWarnings("java:S3011") + private static Object invoke(Object target, String name) { + try { + Method method = target.getClass().getDeclaredMethod(name); + method.setAccessible(true); + return method.invoke(target); + } catch (InvocationTargetException e) { + if (e.getCause() instanceof RuntimeException cause) throw cause; + if (e.getCause() instanceof Error cause) throw cause; + throw new AssertionError("Diagnostic invocation failed: " + name, e.getCause()); + } catch (ReflectiveOperationException e) { + throw new AssertionError("Cannot inspect " + name, e); + } + } +} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryStorageManagerTest.java b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryStorageManagerTest.java deleted file mode 100644 index e4313e7ad..000000000 --- a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryStorageManagerTest.java +++ /dev/null @@ -1,159 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.memory; - -import fr.inria.corese.core.next.storage.api.lifecycle.LifecycleState; -import fr.inria.corese.core.next.storage.api.config.StorageConfig; -import fr.inria.corese.core.next.storage.api.transaction.TransactionManager; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.DisplayName; -import org.junit.jupiter.api.Nested; -import org.junit.jupiter.api.Test; - -import static org.junit.jupiter.api.Assertions.*; - -/** - * Unit tests for MemoryStorageManager. - */ -class MemoryStorageManagerTest { - - private MemoryStorageManager storageManager; - - @BeforeEach - void setUp() { - storageManager = MemoryStorageManager.builder().build(); - } - - @Nested - @DisplayName("Builder tests") - class BuilderTests { - - @Test - @DisplayName("Should build with default settings") - void shouldBuildWithDefaultSettings() { - MemoryStorageManager storage = MemoryStorageManager.builder().build(); - - assertNotNull(storage); - assertNotNull(storage.queries()); - assertNotNull(storage.mutations()); - assertNotNull(storage.metadata()); - assertNotNull(storage.transactions()); - assertNotNull(storage.lifecycle()); - } - } - - @Nested - @DisplayName("Operations getters") - class OperationsGettersTests { - - @Test - @DisplayName("Should return QueryOperations") - void shouldReturnQueryOperations() { - assertNotNull(storageManager.queries()); - assertInstanceOf(MemoryQueryOperations.class, - storageManager.queries()); - } - - @Test - @DisplayName("Should return MutationOperations") - void shouldReturnMutationOperations() { - assertNotNull(storageManager.mutations()); - assertInstanceOf(MemoryMutationOperations.class, - storageManager.mutations()); - } - - @Test - @DisplayName("Should return MetadataOperations") - void shouldReturnMetadataOperations() { - assertNotNull(storageManager.metadata()); - assertInstanceOf(MemoryMetadataOperations.class, - storageManager.metadata()); - } - - @Test - @DisplayName("Should return TransactionManager") - void shouldReturnTransactionManager() { - assertNotNull(storageManager.transactions()); - assertInstanceOf(MemoryTransactionManager.class, - storageManager.transactions()); - } - - @Test - @DisplayName("Should return StorageLifecycle") - void shouldReturnStorageLifecycle() { - assertNotNull(storageManager.lifecycle()); - assertInstanceOf(MemoryLifecycleManager.class, - storageManager.lifecycle()); - } - } - - @Nested - @DisplayName("Lifecycle tests") - class LifecycleTests { - - @Test - @DisplayName("Should start in NOT_INITIALIZED state") - void shouldStartInNotInitializedState() { - assertEquals(LifecycleState.NOT_INITIALIZED, - storageManager.lifecycle().getState()); - } - - @Test - @DisplayName("Should initialize successfully") - void shouldInitializeSuccessfully() { - StorageConfig config = StorageConfig.builder().build(); - - storageManager.lifecycle().initialize(config); - - assertEquals(LifecycleState.RUNNING, - storageManager.lifecycle().getState()); - assertEquals(java.util.Optional.of(config), storageManager.lifecycle().getConfig()); - } - - @Test - @DisplayName("Should shutdown successfully") - void shouldShutdownSuccessfully() { - StorageConfig config = StorageConfig.builder().build(); - storageManager.lifecycle().initialize(config); - - storageManager.lifecycle().shutdown(); - - assertEquals(LifecycleState.SHUTDOWN, - storageManager.lifecycle().getState()); - assertTrue(storageManager.lifecycle().getConfig().isEmpty()); - } - } - - @Nested - @DisplayName("Transaction tests") - class TransactionTests { - - @Test - @DisplayName("Should support transactions") - void shouldSupportTransactions() { - assertTrue(storageManager.transactions().supportsTransactions()); - } - - @Test - @DisplayName("Should begin a transaction") - void shouldBeginTransaction() { - TransactionManager txManager = storageManager.transactions(); - try (var transaction = txManager.beginTransaction()) { - assertTrue(transaction.isActive()); - } - } - } - - @Nested - @DisplayName("toString tests") - class ToStringTests { - - @Test - @DisplayName("Should return meaningful toString") - void shouldReturnMeaningfulToString() { - String result = storageManager.toString(); - - assertNotNull(result); - assertTrue(result.contains("MemoryStorageManager")); - assertTrue(result.contains("size=")); - } - } -} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryStorageTest.java b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryStorageTest.java new file mode 100644 index 000000000..d8de63ca1 --- /dev/null +++ b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryStorageTest.java @@ -0,0 +1,714 @@ +package fr.inria.corese.core.next.storage.impl.memory; + +import fr.inria.corese.core.next.data.Values; +import fr.inria.corese.core.next.data.api.model.Statement; +import fr.inria.corese.core.next.data.api.term.IRI; +import fr.inria.corese.core.next.data.api.term.Resource; +import fr.inria.corese.core.next.data.api.term.SimpleBNode; +import fr.inria.corese.core.next.data.api.term.SimpleIRI; +import fr.inria.corese.core.next.data.api.term.SimpleLiteral; +import fr.inria.corese.core.next.data.api.term.SimpleTriple; +import fr.inria.corese.core.next.data.api.term.Value; +import fr.inria.corese.core.next.query.Repositories; +import fr.inria.corese.core.next.storage.Storages; +import fr.inria.corese.core.next.storage.api.StorageManager; +import fr.inria.corese.core.next.storage.api.config.StorageConfig; +import fr.inria.corese.core.next.storage.api.exception.StorageException; +import fr.inria.corese.core.next.storage.api.model.StatementPattern; +import fr.inria.corese.core.next.storage.api.transaction.IsolationLevel; +import org.junit.jupiter.api.Test; + +import java.util.ArrayList; +import java.util.Arrays; +import java.util.HashSet; +import java.util.Iterator; +import java.util.List; +import java.util.Random; +import java.util.Set; +import java.util.concurrent.CompletableFuture; +import java.util.concurrent.atomic.AtomicInteger; +import java.util.stream.Collectors; + +import static org.junit.jupiter.api.Assertions.*; + +class MemoryStorageTest { + private static final IRI P = new SimpleIRI("urn:p"); + private static final Resource S = new SimpleIRI("urn:s"); + private static final Value O = new SimpleIRI("urn:o"); + private static final Resource G = new SimpleIRI("urn:g"); + private static final StorageConfig CONFIG = StorageConfig.memory(); + + private static Statement statement(int i, Resource graph) { + return Values.factory().createStatement(new SimpleIRI("urn:s" + i), P, new SimpleIRI("urn:o" + i), graph); + } + private static IndexedStatementStore store() { + IndexedStatementStore store = new IndexedStatementStore(); + store.lifecycle().initialize(CONFIG); + return store; + } + + @Test void densePrefixesSurviveRemovalRollbackAndReinsertion() { + var store = store(); + List statements = new ArrayList<>(); + for (int i = 0; i < 40; i++) { + String label = switch (i) { + case 0 -> "Aa"; + case 1 -> "BB"; + default -> "value" + i; + }; + statements.add(Values.factory().createStatement(S, P, new SimpleLiteral(label))); + } + assertEquals(40, store.addAll(statements)); + StatementPattern prefix = StatementPattern.of(S, P, null, (Resource) null, null); + assertEquals(40, store.count(prefix)); + try (var transaction = store.beginTransaction()) { + for (int i = 0; i < 39; i++) assertTrue(store.remove(statements.get(i))); + assertEquals(1, store.count(prefix)); + assertEquals(List.of(statements.getLast()), store.find(prefix).toList()); + transaction.rollback(); + } + assertEquals(40, store.count(prefix)); + for (int i = 0; i < 39; i++) assertTrue(store.remove(statements.get(i))); + assertEquals(1, store.count(prefix)); + assertTrue(store.add(statements.getFirst())); + assertFalse(store.add(statements.getFirst())); + assertEquals(2, store.count(prefix)); + assertTrue(store.remove(statements.getLast())); + assertTrue(store.remove(statements.getFirst())); + assertEquals(0, store.count(prefix)); + assertTrue(store.add(statements.get(1))); + assertEquals(1, store.count(prefix)); + store.closeSession(); + } + + @Test void sharedKeysSurviveCrossRoleAliasesChurnAndBulkMerges() { + var store = store(); + List triples = List.of( + Values.factory().createStatement(S, (IRI) S, S), + Values.factory().createStatement(P, (IRI) S, O), + Values.factory().createStatement((Resource) O, P, S)); + for (Statement triple : triples) { assertTrue(store.add(triple)); MemoryStorageAssertions.verify(store); } + try (var snapshot = store.beginTransaction()) { + for (Statement triple : triples) { assertTrue(store.remove(triple)); MemoryStorageAssertions.verify(store); } + assertEquals(0, store.count(StatementPattern.matchAll())); + snapshot.rollback(); + } + MemoryStorageAssertions.verify(store); + assertEquals(3, store.count(StatementPattern.matchAll())); + for (Statement triple : triples) { assertTrue(store.remove(triple)); MemoryStorageAssertions.verify(store); } + for (int iteration = 0; iteration < 20; iteration++) { + Statement singleton = statement(iteration, null); + assertEquals(1, store.addAll(List.of(singleton))); + MemoryStorageAssertions.verify(store); + assertEquals(3, store.addAll(triples)); + MemoryStorageAssertions.verify(store); + assertTrue(store.remove(singleton)); + MemoryStorageAssertions.verify(store); + for (Statement triple : triples) { assertTrue(store.remove(triple)); MemoryStorageAssertions.verify(store); } + } + store.closeSession(); + } + + @Test void pluginIsExplicitAndDefaultRemainsMemory() { + try (var indexed = Storages.create(CONFIG); var memory = Storages.create()) { + assertInstanceOf(MemoryStorageManager.class, indexed); + assertInstanceOf(fr.inria.corese.core.next.storage.impl.memory.MemoryStorageManager.class, memory); + assertTrue(indexed.isOpen()); + assertEquals(Set.of(IsolationLevel.SERIALIZABLE), indexed.transactions().getSupportedIsolationLevels()); + } + } + + @Test void nativeValuesAreSharedForUnitAndBulkWithoutMergingLexicalVariants() { + for (boolean bulk : List.of(false, true)) { + try (var storage = Storages.create(CONFIG)) { + var rows = List.of( + Values.factory().createStatement(new SimpleIRI("urn:shared"), new SimpleIRI("urn:p1"), new SimpleLiteral("01")), + Values.factory().createStatement(new SimpleIRI("urn:shared"), new SimpleIRI("urn:p2"), new SimpleLiteral("01")), + Values.factory().createStatement(new SimpleIRI("urn:shared"), new SimpleIRI("urn:p3"), new SimpleLiteral("1"))); + if (bulk) assertEquals(3, storage.mutations().addAll(rows)); + else rows.forEach(row -> assertTrue(storage.mutations().add(row))); + List actual = storage.queries().find(StatementPattern.matchAll()).toList(); + assertEquals(new HashSet<>(rows), new HashSet<>(actual)); + assertSame(actual.get(0).getSubject(), actual.get(1).getSubject()); + assertSame(actual.get(0).getObject(), actual.get(1).getObject()); + assertNotSame(actual.get(0).getObject(), actual.get(2).getObject()); + MemoryStorageAssertions.verify((IndexedStatementStore) storage.queries()); + } + } + } + + @Test void copyOnWritePreservesMultipleReadersAcrossWritesBatchesAndGraphDrops() { + try (var root = Storages.create(CONFIG); var a = root.openSession(); var b = root.openSession()) { + List original = List.of(statement(1, null), statement(2, G)); + root.mutations().addAll(original); + try (var txA = a.transactions().beginTransaction(); var txB = b.transactions().beginTransaction()) { + assertTrue(root.mutations().remove(original.getFirst())); + root.mutations().addAll(List.of(statement(3, null), statement(4, G))); + assertTrue(root.mutations().dropGraph(G)); + assertEquals(new HashSet<>(original), new HashSet<>(a.queries().find(StatementPattern.matchAll()).toList())); + assertEquals(new HashSet<>(original), new HashSet<>(b.queries().find(StatementPattern.matchAll()).toList())); + assertTrue(a.mutations().remove(original.getLast())); + assertEquals(List.of(original.getFirst()), a.queries().find(StatementPattern.matchAll()).toList()); + assertEquals(2, b.queries().count(StatementPattern.matchAll())); + assertThrows(StorageException.class, txA::commit); + txA.rollback(); + txB.rollback(); + } + try (var tx = a.transactions().beginTransaction()) { + assertEquals(1, a.mutations().addAll(List.of(statement(5, G)))); + tx.commit(); + } + assertEquals(2, root.queries().count(StatementPattern.matchAll())); + for (var session : List.of(root, a, b)) MemoryStorageAssertions.verify((IndexedStatementStore) session.queries()); + } + } + + @Test void graphContextsAreSharedAcrossUnitsBatchesAndReaderSnapshots() { + for (boolean bulk : List.of(false, true)) { + for (boolean blank : List.of(false, true)) { + testContextSharingScenario(bulk, blank); + } + } + } + + private void testContextSharingScenario(boolean bulk, boolean blank) { + try (var root = Storages.create(CONFIG); var reader = root.openSession()) { + Resource canonical = blank ? new SimpleBNode("context") : new SimpleIRI("urn:context"); + assertTrue(root.mutations().createGraph(canonical)); + for (int round = 0; round < 3; round++) { + List rows = new ArrayList<>(); + for (int i = 0; i < 12; i++) { + rows.add(statement(round * 12 + i, blank ? new SimpleBNode("context") : new SimpleIRI("urn:context"))); + } + if (bulk) { + assertEquals(12, root.mutations().addAll(rows)); + } else { + rows.forEach(row -> assertTrue(root.mutations().add(row))); + } + } + List original = root.queries().find(StatementPattern.matchAll()).toList(); + original.forEach(row -> assertSame(canonical, row.getContext())); + try (var tx = reader.transactions().beginTransaction()) { + assertEquals(36, root.mutations().clear(canonical)); + assertEquals(1, root.mutations().addAll(List.of(statement(100, + blank ? new SimpleBNode("context") : new SimpleIRI("urn:context"))))); + assertSame(canonical, root.queries().find(StatementPattern.matchAll()).findFirst().orElseThrow().getContext()); + assertTrue(root.mutations().dropGraph(canonical)); + assertEquals(original, reader.queries().find(StatementPattern.matchAll()).toList()); + tx.rollback(); + } + MemoryStorageAssertions.verify((IndexedStatementStore) root.queries()); + } + } + + @Test void contextSharingPreservesDifferentIRIViewsForTheSameGraph() { + for (boolean bulk : List.of(false, true)) { + try (var root = Storages.create(CONFIG)) { + IRI first = new SimpleIRI("urn:", "context:graph"); + IRI second = new SimpleIRI("urn:context:", "graph"); + List input = List.of(statement(1, first), statement(2, second)); + if (bulk) root.mutations().addAll(input); + else input.forEach(root.mutations()::add); + List actual = root.queries().find(StatementPattern.matchAll()).toList(); + assertEquals(1, root.metadata().getContexts().size()); + assertEquals(input, actual); + assertEquals("urn:", ((IRI) actual.get(0).getContext()).getNamespace()); + assertEquals("urn:context:", ((IRI) actual.get(1).getContext()).getNamespace()); + } + } + } + + @Test void inlineLeafTransitionsPreserveLexicalIdentityAndCopiedPermutations() { + for (TripleIndex.Order order : TripleIndex.Order.values()) { + TripleIndex index = new TripleIndex(order); + List rows = new ArrayList<>(); + for (String suffix : List.of("Aa", "BB", "01", "1", "2", "3", "4", "5", "6", "7", "8", "9")) { + rows.add(switch (order) { + case SPO -> Values.factory().createStatement(S, P, new SimpleLiteral(suffix)); + case POS -> Values.factory().createStatement(new SimpleIRI("urn:" + suffix), P, O); + case OSP -> Values.factory().createStatement(S, new SimpleIRI("urn:" + suffix), O); + }); + } + for (Statement row : rows) { assertTrue(MemoryStorageAssertions.add(index, row)); assertFalse(MemoryStorageAssertions.add(index, row)); } + TripleIndex copy = index.copy(); + for (int i = 0; i < rows.size() - 1; i++) assertTrue(MemoryStorageAssertions.remove(index, rows.get(i))); + assertEquals(1, index.size()); + assertTrue(MemoryStorageAssertions.add(index, rows.getFirst())); + assertEquals(2, index.size()); + assertEquals(rows.size(), copy.size()); + for (Statement row : rows) { + StatementPattern pattern = StatementPattern.of(row.getSubject(), row.getPredicate(), row.getObject()); + assertEquals(1, copy.count(pattern)); + assertEquals(row == rows.getLast() || row == rows.getFirst() ? 1 : 0, index.count(pattern)); + } + assertTrue(MemoryStorageAssertions.remove(index, rows.getFirst())); + assertTrue(MemoryStorageAssertions.remove(index, rows.getLast())); + assertEquals(0, index.size()); + assertTrue(MemoryStorageAssertions.add(index, rows.get(1))); + } + } + + @Test void poolingPreservesIRIViewsAndExplicitLiteralMetadata() { + for (boolean bulk : List.of(false, true)) { + try (var storage = Storages.create(CONFIG)) { + IRI first = new SimpleIRI("urn:", "view:item"); + IRI second = new SimpleIRI("urn:view:", "item"); + assertEquals(first.stringValue(), second.stringValue()); + IRI datatype = new SimpleIRI("urn:custom-datatype"); + SimpleLiteral text = new SimpleLiteral("10", datatype, + fr.inria.corese.core.next.data.api.literal.XSDDatatype.STRING); + SimpleLiteral number = new SimpleLiteral("10", datatype, + fr.inria.corese.core.next.data.api.literal.XSDDatatype.INTEGER); + SimpleLiteral firstDatatypeView = new SimpleLiteral("20", new SimpleIRI("urn:", "datatype:x")); + SimpleLiteral secondDatatypeView = new SimpleLiteral("20", new SimpleIRI("urn:datatype:", "x")); + List input = List.of( + Values.factory().createStatement(first, new SimpleIRI("urn:p1"), text), + Values.factory().createStatement(second, new SimpleIRI("urn:p2"), number), + Values.factory().createStatement(first, new SimpleIRI("urn:p3"), firstDatatypeView), + Values.factory().createStatement(first, new SimpleIRI("urn:p4"), secondDatatypeView)); + if (bulk) assertEquals(4, storage.mutations().addAll(input)); + else input.forEach(row -> assertTrue(storage.mutations().add(row))); + List actual = storage.queries().find(StatementPattern.matchAll()).toList(); + assertEquals(input, actual); + assertEquals(first.getNamespace(), ((IRI) actual.get(0).getSubject()).getNamespace()); + assertEquals(second.getNamespace(), ((IRI) actual.get(1).getSubject()).getNamespace()); + assertSame(text.getCoreDatatype(), ((SimpleLiteral) actual.get(0).getObject()).getCoreDatatype()); + assertSame(number.getCoreDatatype(), ((SimpleLiteral) actual.get(1).getObject()).getCoreDatatype()); + assertEquals(firstDatatypeView.getDatatype().getNamespace(), ((SimpleLiteral) actual.get(2).getObject()).getDatatype().getNamespace()); + assertEquals(secondDatatypeView.getDatatype().getNamespace(), ((SimpleLiteral) actual.get(3).getObject()).getDatatype().getNamespace()); + MemoryStorageAssertions.verify((IndexedStatementStore) storage.queries()); + } + } + } + + @SuppressWarnings("java:S2245") + @Test void allMasksAndContextSelectionsMatchIndependentQuadOracleAfterRandomMutations() { + var store = store(); + Random random = new Random(614); + Set oracle = new HashSet<>(); + Set registeredGraphs = new java.util.LinkedHashSet<>(); + List contexts = Arrays.asList(null, G, new SimpleBNode("g")); + List pool = new ArrayList<>(); + for (Resource context : contexts) { + for (int i = 0; i < 12; i++) { + pool.add(Values.factory().createStatement(new SimpleIRI("urn:s" + i % 3), + new SimpleIRI("urn:p" + i % 2), new SimpleLiteral("v" + i % 4), context)); + } + } + for (int operation = 0; operation < 120; operation++) { + Statement s = applyRandomMutation(store, oracle, pool, random); + MemoryStorageAssertions.verify(store); + oracle.stream().map(Statement::getContext).filter(java.util.Objects::nonNull).forEach(registeredGraphs::add); + assertEquals(registeredGraphs, store.getContexts()); + Set subjects = oracle.stream().map(Statement::getSubject).collect(Collectors.toSet()); + Set predicates = oracle.stream().map(Statement::getPredicate).collect(Collectors.toSet()); + Set objects = oracle.stream().map(Statement::getObject).collect(Collectors.toSet()); + assertEquals(subjects, store.getSubjects()); + assertEquals(predicates, store.getPredicates()); + assertEquals(objects, store.getObjects()); + assertEquals(new fr.inria.corese.core.next.storage.api.model.StorageStatistics( + oracle.size(), subjects.size(), predicates.size(), objects.size(), registeredGraphs.size()), + store.getStatistics()); + verifyMasksAndContexts(store, oracle, s); + } + } + + private static Statement applyRandomMutation(IndexedStatementStore store, Set oracle, + List pool, Random random) { + Statement s = pool.get(random.nextInt(pool.size())); + switch (random.nextInt(4)) { + case 0 -> assertEquals(oracle.add(s), store.add(s)); + case 1 -> assertEquals(oracle.remove(s), store.remove(s)); + case 2 -> { + List batch = List.of(s, pool.get(random.nextInt(pool.size())), s); + long added = batch.stream().filter(oracle::add).count(); + assertEquals(added, store.addAll(batch)); + } + default -> { + StatementPattern pattern = StatementPattern.of(s.getSubject(), null, null, s.getContext()); + long removed = oracle.stream().filter(t -> matches(t, pattern)).count(); + oracle.removeIf(t -> matches(t, pattern)); + assertEquals(removed, store.remove(pattern)); + } + } + return s; + } + + private static void verifyMasksAndContexts(IndexedStatementStore store, Set oracle, Statement s) { + for (int mask = 0; mask < 8; mask++) { + for (Resource[] selected : List.of(new Resource[0], new Resource[]{null}, new Resource[]{G}, + new Resource[]{null, G, G}, new Resource[]{new SimpleIRI("urn:absent")})) { + StatementPattern p = StatementPattern.of((mask & 1) != 0 ? s.getSubject() : null, + (mask & 2) != 0 ? s.getPredicate() : null, (mask & 4) != 0 ? s.getObject() : null, selected); + Set expected = oracle.stream().filter(t -> matches(t, p)).collect(Collectors.toSet()); + try (var found = store.find(p)) { assertEquals(expected, found.collect(Collectors.toSet())); } + assertEquals(expected.size(), store.count(p)); + assertEquals(!expected.isEmpty(), store.contains(p)); + } + } + } + private static boolean matches(Statement s, StatementPattern p) { + return (p.getSubject() == null || p.getSubject().equals(s.getSubject())) + && (p.getPredicate() == null || p.getPredicate().equals(s.getPredicate())) + && (p.getObject() == null || p.getObject().equals(s.getObject())) + && (p.isContextWildcard() || Arrays.asList(p.getContexts()).contains(s.getContext())); + } + + @Test void everyPermutationVisitsOnlyItsMatchingLeaves() { + for (TripleIndex.Order order : TripleIndex.Order.values()) { + TripleIndex index = new TripleIndex(order); + for (int i = 0; i < 1000; i++) MemoryStorageAssertions.add(index, statement(i, null)); + Statement s = statement(42, null); + StatementPattern p = switch (order) { + case SPO -> StatementPattern.of(s.getSubject(), P, null); + case POS -> StatementPattern.of(null, P, s.getObject()); + case OSP -> StatementPattern.of(s.getSubject(), null, s.getObject()); + }; + AtomicInteger visits = new AtomicInteger(); + index.visit(p, candidate -> { visits.incrementAndGet(); assertEquals(s, candidate); return false; }); + assertEquals(1, visits.get()); + } + } + + @Test void multiGraphSearchesKeepRdfIdentityAndSnapshotsAcrossBoundRemoval() { + IRI integer = new SimpleIRI("http://www.w3.org/2001/XMLSchema#integer"); + List input = new ArrayList<>(); + List graphs = new ArrayList<>(); + graphs.add(null); + for (int i = 0; i < 128; i++) graphs.add(new SimpleIRI("urn:search:graph:" + i)); + Resource left = new SimpleIRI("urn:", "Aa"); + Resource right = new SimpleIRI("urn:BB"); + // Colliding subject hashes, lexical numeric variants and nested triple objects. + for (Resource graph : graphs) { + input.add(Values.factory().createStatement(left, P, new SimpleLiteral("01", integer), graph)); + input.add(Values.factory().createStatement(left, P, new SimpleLiteral("1", integer), graph)); + input.add(Values.factory().createStatement(right, P, new SimpleLiteral("01", integer), graph)); + input.add(Values.factory().createStatement(new SimpleBNode("Aa"), new SimpleIRI("urn:alt"), + new SimpleTriple(left, P, right), graph)); + } + try (var root = Storages.create(CONFIG); var reader = root.openSession()) { + root.mutations().addAll(input); + StatementPattern remove = StatementPattern.of(new SimpleIRI("urn:Aa"), P, new SimpleLiteral("01", integer)); + try (var transaction = reader.transactions().beginTransaction()) { + assertEquals(graphs.size(), root.mutations().remove(remove)); + for (var session : List.of(root, reader)) { + List expectedRows = session == root + ? input.stream().filter(s -> !matches(s, remove)).toList() : input; + verifySessionPatternSearches(session, expectedRows, input.subList(0, 4), graphs); + MemoryStorageAssertions.verify((IndexedStatementStore) session.queries()); + } + transaction.rollback(); + } + assertEquals(0, reader.queries().count(remove)); + assertEquals(graphs.size(), root.mutations().addAll(input)); + assertEquals(graphs.size(), reader.queries().count(remove)); + } + } + + private static void verifySessionPatternSearches(StorageManager session, List expectedRows, + List sampleTargets, List graphs) { + for (Statement target : sampleTargets) { + for (int mask = 0; mask < 8; mask++) { + verifyMaskAcrossContexts(session, expectedRows, target, mask, graphs); + } + } + } + + private static void verifyMaskAcrossContexts(StorageManager session, List expectedRows, + Statement target, int mask, List graphs) { + Resource s = (mask & 1) != 0 ? target.getSubject() : null; + IRI p = (mask & 2) != 0 ? target.getPredicate() : null; + Value o = (mask & 4) != 0 ? target.getObject() : null; + for (Resource[] contexts : List.of(new Resource[0], new Resource[]{null}, + new Resource[]{graphs.getLast(), graphs.get(1), null, graphs.get(1)}, + new Resource[]{new SimpleIRI("urn:missing-graph")})) { + StatementPattern pattern = StatementPattern.of(s, p, o, contexts); + Set expected = expectedRows.stream().filter(row -> matches(row, pattern)) + .collect(Collectors.toSet()); + assertEquals(expected.size(), session.queries().count(pattern)); + assertEquals(!expected.isEmpty(), session.queries().contains(pattern)); + try (var found = session.queries().find(pattern)) { + assertEquals(expected, new HashSet<>(found.toList())); + } + } + } + + @Test void graphNamesSurviveClearAndLastRemovalButNotDrop() { + var store = store(); + assertTrue(store.createGraph(G)); + assertFalse(store.createGraph(G)); + assertEquals(Set.of(G), store.getContexts()); + store.add(statement(1, G)); + store.add(statement(1, null)); + assertEquals(1, store.clear((Resource) null)); + assertEquals(1, store.clear(G)); + assertEquals(Set.of(G), store.getContexts()); + store.add(statement(1, G)); + assertTrue(store.remove(statement(1, G))); + assertFalse(store.remove(statement(1, G))); + assertEquals(Set.of(G), store.getContexts()); + assertTrue(store.dropGraph(G)); + assertFalse(store.dropGraph(G)); + assertEquals(0, store.getStatistics().contextCount()); + MemoryStorageAssertions.verify(store); + assertThrows(UnsupportedOperationException.class, () -> store.getContexts().add(G)); + } + + @Test void termIdentityAndNestedTriplesArePreserved() { + var store = store(); + IRI integer = new SimpleIRI("http://www.w3.org/2001/XMLSchema#integer"); + List objects = List.of(new SimpleLiteral("01", integer), new SimpleLiteral("1", integer), + new SimpleIRI("urn:same"), new SimpleBNode("urn:same"), + new SimpleLiteral("word", "en"), new SimpleLiteral("word", "fr"), new SimpleTriple(S, P, O)); + for (Value o : objects) assertTrue(store.add(Values.factory().createStatement(S, P, o))); + assertFalse(store.add(Values.factory().createStatement(S, P, new SimpleLiteral("word", "EN")))); + assertEquals(objects.size(), store.count(StatementPattern.matchAll())); + for (Value o : objects) assertEquals(1, store.count(StatementPattern.of(S, P, o))); + MemoryStorageAssertions.verify(store); + } + + @Test void streamsSnapshotMatchesAndContextFilterOrderDoesNotAffectTraversal() { + var store = store(); + store.add(statement(1, G)); store.add(statement(2, null)); + List first; + try (var stream = store.find(StatementPattern.of(null, null, null, G, null))) { first = stream.toList(); } + try (var stream = store.find(StatementPattern.of(null, null, null, null, G))) { assertEquals(first, stream.toList()); } + try (var stream = store.find(StatementPattern.matchAll())) { + store.clear(); + assertEquals(first, stream.toList()); + } + } + + @Test void batchIsSingleUseAndCountsOnlyNewQuads() { + var store = store(); + store.add(statement(1, G)); + AtomicInteger iterators = new AtomicInteger(); + Iterable once = () -> { + assertEquals(1, iterators.incrementAndGet()); + return List.of(statement(1, G), statement(2, G), statement(2, G), statement(2, null)).iterator(); + }; + assertEquals(2, store.addAll(once)); + assertEquals(3, store.count(StatementPattern.matchAll())); + assertEquals(0, store.addAll(List.of())); + MemoryStorageAssertions.verify(store); + } + + @Test void sourceFailuresNeverPublishAPrefixOrNewGraph() { + var store = store(); + for (String phase : List.of("iterator", "hasNext", "next", "null")) { + Iterable bad = createFailingIterable(phase); + Class expected = "null".equals(phase) ? NullPointerException.class : StorageException.class; + assertThrows(expected, () -> store.addAll(bad)); + assertEquals(0, store.count(StatementPattern.matchAll())); + assertTrue(store.getContexts().isEmpty()); + MemoryStorageAssertions.verify(store); + } + assertTrue(store.add(statement(2, null))); + } + + private static Iterable createFailingIterable(String phase) { + if ("iterator".equals(phase)) { + return () -> { throw new IllegalArgumentException("source failure"); }; + } + return () -> new Iterator<>() { + int position; + @Override public boolean hasNext() { + if (position == 1 && "hasNext".equals(phase)) { + throw new IllegalArgumentException("source failure"); + } + return position < 2; + } + @Override public Statement next() { + if (position++ == 0) return statement(1, G); + if ("null".equals(phase)) return null; + throw new IllegalArgumentException("source failure"); + } + }; + } + + @Test void sessionsOnSameThreadHaveIndependentSnapshotsAndGlobalConflicts() { + try (StorageManager root = Storages.create(CONFIG)) { + StorageManager a = root.openSession(); + StorageManager b = root.openSession(); + try (var txA = a.transactions().beginTransaction(); var txB = b.transactions().beginTransaction()) { + a.mutations().add(statement(1, G)); + b.mutations().createGraph(new SimpleIRI("urn:empty")); + assertEquals(0, b.queries().count(StatementPattern.matchAll())); + assertTrue(root.metadata().getContexts().isEmpty()); + txA.commit(); + assertEquals(0, b.queries().count(StatementPattern.matchAll())); + assertThrows(StorageException.class, txB::commit); + assertTrue(txB.isActive()); + txB.rollback(); + } + assertEquals(Set.of(G), root.metadata().getContexts()); + assertEquals(1, b.queries().count(StatementPattern.matchAll())); + a.close(); + assertTrue(root.isOpen()); + assertFalse(a.isOpen()); + } + } + + @Test void otherThreadsCannotAccidentallyAutocommitIntoAnActiveSession() { + try (var root = Storages.create(CONFIG); var session = root.openSession(); var tx = session.transactions().beginTransaction()) { + CompletableFuture.runAsync(() -> { + assertThrows(IllegalStateException.class, () -> session.mutations().add(statement(1, null))); + assertThrows(IllegalStateException.class, () -> session.queries().count(StatementPattern.matchAll())); + assertThrows(IllegalStateException.class, tx::commit); + }).join(); + CompletableFuture.runAsync(() -> { + try (var other = root.openSession()) { other.mutations().add(statement(2, G)); } + }).join(); + assertEquals(0, session.queries().count(StatementPattern.matchAll())); + assertThrows(StorageException.class, tx::commit); + } + } + + @Test void batchFailurePreservesEarlierTransactionWritesAndRollbackDiscardsGraphs() { + try (var root = Storages.create(CONFIG); var session = root.openSession()) { + try (var tx = session.transactions().beginTransaction()) { + session.mutations().add(statement(1, null)); + assertThrows(NullPointerException.class, () -> session.mutations().addAll(Arrays.asList(statement(2, G), null))); + assertEquals(1, session.queries().count(StatementPattern.matchAll())); + assertTrue(session.metadata().getContexts().isEmpty()); + assertEquals(1, session.mutations().addAll(List.of(statement(2, G)))); + assertEquals(0, root.queries().count(StatementPattern.matchAll())); + tx.rollback(); + } + assertEquals(0, root.queries().count(StatementPattern.matchAll())); + assertTrue(root.metadata().getContexts().isEmpty()); + } + } + + @Test void batchSourceCanCommitAnotherSessionWithoutLosingItsChanges() { + try (var root = Storages.create(CONFIG); var a = root.openSession(); var b = root.openSession()) { + Iterable source = () -> { + assertThrows(IllegalStateException.class, () -> a.mutations().add(statement(9, null))); + b.mutations().add(statement(1, G)); + return List.of(statement(2, null)).iterator(); + }; + assertEquals(1, a.mutations().addAll(source)); + assertEquals(2, root.queries().count(StatementPattern.matchAll())); + } + } + + @Test void publicQueriesUpdatesBindingsAndEmptyGraphsUseTheNewBackend() { + try (var repository = Repositories.create(CONFIG); var a = repository.getConnection(); var b = repository.getConnection()) { + a.add(List.of(Values.factory().createStatement(S, P, O), + Values.factory().createStatement((Resource) O, new SimpleIRI("urn:name"), new SimpleLiteral("Bob")))); + assertTrue(a.prepareBooleanQuery("ASK { ?friend . ?friend \"Bob\" }").evaluate()); + assertFalse(a.prepareBooleanQuery("ASK { ?same ?same }").evaluate()); + a.begin(); + a.prepareUpdate("CREATE GRAPH ; INSERT DATA { GRAPH { } }").execute(); + assertTrue(a.prepareBooleanQuery("ASK { GRAPH {} }").evaluate()); + assertFalse(b.prepareBooleanQuery("ASK { GRAPH { ?s ?p ?o } }").evaluate()); + a.commit(); + assertTrue(b.prepareBooleanQuery("ASK { GRAPH { ?s ?p ?o } }").evaluate()); + assertFalse(b.prepareBooleanQuery("ASK { ?o }").evaluate()); + } + } + @Test void numericAndTemporalValuesKeepTheirEvaluationCapabilities() { + try (var repository = Repositories.create(CONFIG); var connection = repository.getConnection()) { + connection.add(List.of(Values.factory().createStatement(S, P, Values.factory().createLiteral(30)), + Values.factory().createStatement(S, new SimpleIRI("urn:date"), Values.factory().createLiteral( + "2026-10-02T12:30:00Z", new SimpleIRI("http://www.w3.org/2001/XMLSchema#dateTime"))))); + assertTrue(connection.prepareBooleanQuery("ASK { ?s ?age FILTER(?age + 1 = 31) }").evaluate()); + assertTrue(connection.prepareBooleanQuery("ASK { ?s ?date FILTER(HOURS(?date) = 12) }").evaluate()); + } + } + + @Test void foreignTermImplementationsAreCapturedAndSearchKeysAreNormalized() { + var store = store(); + var external = new fr.inria.corese.core.next.data.spi.term.AbstractIRI("urn:s") { + @Override public boolean equals(Object o) { return super.equals(o); } + @Override public int hashCode() { return stringValue().hashCode(); } + }; + store.add(Values.factory().createStatement(external, P, O)); + assertTrue(store.contains(StatementPattern.of(S, P, O))); + assertTrue(store.contains(StatementPattern.of(external, P, O))); + assertEquals(1, store.remove(StatementPattern.of(external, null, null))); + MemoryStorageAssertions.verify(store); + } + + @Test void collidingHashesStillDistinguishRdfTermsAndLiteralLexicalForms() { + var store = store(); + List objects = List.of(new SimpleIRI("urn:Aa"), new SimpleIRI("urn:BB"), + new SimpleBNode("urn:Aa"), new SimpleLiteral("Aa"), new SimpleLiteral("BB"), + Values.factory().createLiteral("01", new SimpleIRI("http://www.w3.org/2001/XMLSchema#integer")), + Values.factory().createLiteral("1", new SimpleIRI("http://www.w3.org/2001/XMLSchema#integer"))); + List statements = objects.stream().map(v -> Values.factory().createStatement(S, P, v)).toList(); + assertEquals(statements.size(), store.addAll(statements)); + for (Value value : objects) assertEquals(1, store.count(StatementPattern.of(null, null, value))); + try (var tx = store.beginTransaction()) { + assertEquals(statements.size(), store.count(StatementPattern.matchAll())); + assertTrue(store.remove(statements.getFirst())); + tx.rollback(); + } + assertEquals(statements.size(), store.count(StatementPattern.matchAll())); + MemoryStorageAssertions.verify(store); + } + + @Test void publicJoinPushesBindingsToTheIndexedSession() { + try (var root = Storages.create(CONFIG)) { + root.mutations().addAll(List.of(Values.factory().createStatement(S, P, O), + Values.factory().createStatement((Resource) O, new SimpleIRI("urn:name"), new SimpleLiteral("name")))); + var session = org.mockito.Mockito.spy(root.openSession()); + var queries = org.mockito.Mockito.spy(session.queries()); + List patterns = new ArrayList<>(); + org.mockito.Mockito.doAnswer(invocation -> { + patterns.add(invocation.getArgument(0)); + return invocation.callRealMethod(); + }).when(queries).find(org.mockito.ArgumentMatchers.any()); + org.mockito.Mockito.doReturn(queries).when(session).queries(); + var owner = org.mockito.Mockito.spy(root); + org.mockito.Mockito.doReturn(session).when(owner).openSession(); + try (var repository = Repositories.create(owner); var connection = repository.getConnection()) { + assertTrue(connection.prepareBooleanQuery( + "ASK { VALUES ?s { } ?s ?friend . ?friend ?name }").evaluate()); + } + assertTrue(patterns.stream().anyMatch(p -> S.equals(p.getSubject()) && P.equals(p.getPredicate()))); + // The optimizer may reverse the join: either subject or object carries its binding. + assertTrue(patterns.stream().anyMatch(p -> + (O.equals(p.getSubject()) && new SimpleIRI("urn:name").equals(p.getPredicate())) + || (S.equals(p.getSubject()) && P.equals(p.getPredicate()) && O.equals(p.getObject())))); + } + } + + @Test void concurrentOwnerCloseIsIdempotentAndExistingStreamsRemainUsable() { + var root = Storages.create(CONFIG); + root.mutations().add(statement(1, null)); + try (var snapshot = root.queries().find(StatementPattern.matchAll())) { + var gate = new java.util.concurrent.CyclicBarrier(2); + Runnable close = () -> { + try { + gate.await(); + } catch (InterruptedException failure) { + Thread.currentThread().interrupt(); + throw new AssertionError(failure); + } catch (Exception failure) { + throw new AssertionError(failure); + } + root.close(); + }; + CompletableFuture.allOf(CompletableFuture.runAsync(close), CompletableFuture.runAsync(close)).join(); + assertFalse(root.isOpen()); + assertEquals(1, snapshot.count()); + assertThrows(IllegalStateException.class, () -> root.queries().count(StatementPattern.matchAll())); + root.close(); + } + } + + @Test void literalsCaptureForeignDatatypeIdentityEvenInsideBuiltinLiteral() { + var datatype = new java.util.concurrent.atomic.AtomicReference<>("urn:original"); + var external = new fr.inria.corese.core.next.data.spi.term.AbstractIRI("urn:original") { + @Override public String stringValue() { return datatype.get(); } + }; + var store = store(); + store.add(Values.factory().createStatement(S, P, new SimpleLiteral("value", external))); + datatype.set("urn:changed"); + assertTrue(store.contains(StatementPattern.of(S, P, new SimpleLiteral("value", new SimpleIRI("urn:original"))))); + assertFalse(store.contains(StatementPattern.of(S, P, new SimpleLiteral("value", new SimpleIRI("urn:changed"))))); + MemoryStorageAssertions.verify(store); + } + +} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryTransactionManagerTest.java b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryTransactionManagerTest.java deleted file mode 100644 index 2f7e2b583..000000000 --- a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/MemoryTransactionManagerTest.java +++ /dev/null @@ -1,77 +0,0 @@ -package fr.inria.corese.core.next.storage.impl.memory; - -import fr.inria.corese.core.next.data.Values; -import fr.inria.corese.core.next.data.api.model.Statement; -import fr.inria.corese.core.next.storage.api.transaction.IsolationLevel; -import fr.inria.corese.core.next.storage.api.transaction.Transaction; -import org.junit.jupiter.api.BeforeEach; -import org.junit.jupiter.api.Test; - -import java.util.Set; -import java.util.concurrent.CompletableFuture; - -import static org.junit.jupiter.api.Assertions.assertEquals; -import static org.junit.jupiter.api.Assertions.assertFalse; -import static org.junit.jupiter.api.Assertions.assertThrows; -import static org.junit.jupiter.api.Assertions.assertTrue; - -class MemoryTransactionManagerTest { - private InMemoryStatementStore store; - private MemoryTransactionManager transactions; - private Statement statement; - - @BeforeEach - void setUp() { - store = new InMemoryStatementStore(); - transactions = new MemoryTransactionManager(store); - var factory = Values.factory(); - statement = factory.createStatement(factory.createIRI("urn:s"), factory.createIRI("urn:p"), - factory.createIRI("urn:o"), factory.createIRI("urn:g")); - } - - @Test - void commitPublishesStatementsAndGraphNamesTogether() { - try (Transaction transaction = transactions.beginTransaction()) { - store.add(statement); - assertEquals(1, store.size()); - assertEquals(0, CompletableFuture.supplyAsync(store::size).join()); - transaction.commit(); - } - assertEquals(1, store.size()); - assertEquals(Set.of(statement.getContext()), store.getContexts()); - } - - @Test - void closeRollsBackIncludingEmptyGraphNames() { - try (Transaction transaction = transactions.beginTransaction()) { - assertTrue(transaction.isActive()); - store.createGraph(statement.getContext()); - store.add(statement); - } - assertEquals(0, store.size()); - assertTrue(store.getContexts().isEmpty()); - } - - @Test - void concurrentCommitIsDetectedWithoutLosingTheOtherWriter() { - try (Transaction transaction = transactions.beginTransaction()) { - CompletableFuture.runAsync(() -> store.add(statement)).join(); - assertEquals(0, store.size()); - assertThrows(IllegalStateException.class, transaction::commit); - } - assertEquals(1, store.size()); - } - - @Test - void transactionLifecycleAndIsolationAreExplicit() { - assertTrue(transactions.supportsTransactions()); - assertEquals(Set.of(IsolationLevel.SERIALIZABLE), transactions.getSupportedIsolationLevels()); - assertThrows(IllegalArgumentException.class, () -> transactions.beginTransaction(IsolationLevel.READ_UNCOMMITTED)); - try (Transaction transaction = transactions.beginTransaction()) { - assertThrows(IllegalStateException.class, transactions::beginTransaction); - transaction.rollback(); - assertFalse(transaction.isActive()); - assertThrows(IllegalStateException.class, transaction::commit); - } - } -} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/ProducerDeduplicationTest.java b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/ProducerDeduplicationTest.java new file mode 100644 index 000000000..cfdfb6530 --- /dev/null +++ b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/ProducerDeduplicationTest.java @@ -0,0 +1,194 @@ +package fr.inria.corese.core.next.storage.impl.memory; + +import fr.inria.corese.core.next.data.Values; +import fr.inria.corese.core.next.data.api.model.Statement; +import fr.inria.corese.core.next.data.api.term.Resource; +import fr.inria.corese.core.next.data.api.term.SimpleIRI; +import fr.inria.corese.core.next.query.impl.engine.model.Edge; +import fr.inria.corese.core.next.query.impl.engine.model.NodeImpl; +import fr.inria.corese.core.next.query.impl.engine.pattern.Query; +import fr.inria.corese.core.next.query.impl.engine.spi.Environment; +import fr.inria.corese.core.next.query.impl.engine.storage.StorageManagerEdge; +import fr.inria.corese.core.next.query.impl.engine.storage.StorageManagerProducer; +import fr.inria.corese.core.next.query.impl.sparql.ast.SelectQueryAst; +import fr.inria.corese.core.next.query.impl.sparql.bridge.CoreseAstQueryBuilder; +import fr.inria.corese.core.next.query.impl.sparql.execution.NextSparqlPipelineExecutor; +import fr.inria.corese.core.next.query.impl.sparql.parser.SparqlParser; +import fr.inria.corese.core.next.storage.Storages; +import fr.inria.corese.core.next.storage.api.StorageManager; +import fr.inria.corese.core.next.storage.api.config.StorageConfig; +import fr.inria.corese.core.next.storage.api.model.StatementPattern; +import fr.inria.corese.core.next.storage.api.operations.QueryOperations; +import fr.inria.corese.core.next.storage.api.operations.StatementSpliterator; +import org.junit.jupiter.api.Test; +import java.util.ArrayList; +import java.util.Iterator; +import java.util.LinkedHashSet; +import java.util.List; +import java.util.stream.Stream; +import static org.junit.jupiter.api.Assertions.*; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.*; + +class ProducerDeduplicationTest { + private static final Resource G = new SimpleIRI("urn:g"); + private static final Resource H = new SimpleIRI("urn:h"); + private static final Resource EMPTY = new SimpleIRI("urn:empty"); + private static StorageManager storage() { return Storages.create(StorageConfig.memory()); } + private static Statement row(int i, Resource graph) { + return Values.factory().createStatement(new SimpleIRI("urn:s" + i), new SimpleIRI("urn:p"), new SimpleIRI("urn:o"), graph); + } + private static Query query(String dataset) { + return new CoreseAstQueryBuilder().toNextQuery((SelectQueryAst) new SparqlParser() + .parse("SELECT * " + dataset + " WHERE { ?s ?p ?o }")); + } + private static Iterable edges(StorageManagerProducer producer, Query query) { + Environment env = mock(Environment.class); when(env.getQuery()).thenReturn(query); + return producer.getEdges(null, query.getFrom(), query.getBody().first().first().getEdge(), env); + } + @SuppressWarnings("java:S3011") + private static Object seen(Iterator iterator) throws ReflectiveOperationException { + var field = iterator.getClass().getDeclaredField("seen"); + field.setAccessible(true); + return field.get(iterator); + } + private static List rows(Iterator iterator) { + List result = new ArrayList<>(); + while (iterator.hasNext()) result.add(((StorageManagerEdge) iterator.next()).getSourceStatement()); + return result; + } + private static StorageManager observed(StorageManager root, java.util.function.Function> find) { + StorageManager wrapper = mock(StorageManager.class); + QueryOperations queries = mock(QueryOperations.class); + when(wrapper.queries()).thenReturn(queries); when(wrapper.metadata()).thenReturn(root.metadata()); + when(queries.find(any())).thenAnswer(call -> find.apply(call.getArgument(0))); return wrapper; + } + private static StatementSpliterator traversal(Stream stream) { + return assertInstanceOf(StatementSpliterator.class, stream.spliterator()); + } + + @Test void uniquenessMetadataReflectsTheSelectedSnapshotAndIgnoresEmptyPartitions() { + try (var root = storage()) { + root.mutations().add(row(0, null)); root.mutations().add(row(1, G)); root.mutations().createGraph(EMPTY); + try (var all = root.queries().find(StatementPattern.matchAll()); + var one = root.queries().find(StatementPattern.of(null, null, null, G, EMPTY)); + var absent = root.queries().find(StatementPattern.of(null, null, null, H))) { + var multi = traversal(all); + var single = traversal(one); + assertFalse(multi.hasDistinctTriples()); assertTrue(single.hasDistinctTriples()); + assertTrue(traversal(absent).hasDistinctTriples()); + root.mutations().clear(); root.mutations().add(row(2, H)); + assertFalse(multi.hasDistinctTriples()); assertTrue(single.hasDistinctTriples()); + List old = new ArrayList<>(); multi.forEachRemaining(old::add); + assertEquals(List.of(row(0, null), row(1, G)), old); + old.clear(); single.forEachRemaining(old::add); assertEquals(List.of(row(1, G)), old); + } + } + } + + @Test void singleDefaultExplicitFromAndNamedGraphAllocateNoDeduplicationSet() throws Exception { + try (var root = storage()) { + root.mutations().add(row(0, null)); root.mutations().add(row(1, G)); root.mutations().add(row(1, H)); + var producer = new StorageManagerProducer(root); + for (String dataset : List.of("", "FROM ", "FROM FROM ")) { + var q = query(dataset); + var iterable = edges(producer, q); + var iterator = iterable.iterator(); + try { assertNull(seen(iterator)); assertEquals(1, rows(iterator).size()); } + finally { producer.closeEdges(iterable); } + } + var q = query(""); + var iterable = producer.getEdges(NodeImpl.forValue(G), List.of(), q.getBody().first().first().getEdge(), null); + var iterator = iterable.iterator(); + try { assertNull(seen(iterator)); assertEquals(List.of(row(1, G)), rows(iterator)); } + finally { producer.closeEdges(iterable); producer.close(); } + } + } + + @Test void multipleFromContextsWithOnlyOnePopulatedPartitionNeedNoSet() throws Exception { + try (var root = storage()) { + root.mutations().add(row(0, G)); root.mutations().add(row(1, H)); root.mutations().createGraph(EMPTY); + var producer = new StorageManagerProducer(root); + var iterable = edges(producer, query("FROM FROM FROM ")); + var iterator = iterable.iterator(); try { assertNull(seen(iterator)); assertEquals(List.of(row(0, G)), rows(iterator)); } + finally { producer.closeEdges(iterable); producer.close(); } + } + } + + @Test void mergedContextsDeduplicateTriplesInFirstEncounterOrderWithoutLosingStatements() throws Exception { + try (var root = storage()) { + // Ordered roots, a removal hole, and overlapping triples in both named graphs. + for (Resource graph : new Resource[]{G, H}) for (int i = 0; i < 12; i++) root.mutations().add(row(i, graph)); + root.mutations().remove(row(4, G)); root.mutations().add(row(20, H)); + List oracle = new ArrayList<>(); + var keys = new LinkedHashSet>(); + try (var stream = root.queries().find(StatementPattern.of(null, null, null, G, H))) { + stream.forEach(s -> { if (keys.add(List.of(s.getSubject(), s.getPredicate(), s.getObject()))) oracle.add(s); }); + } + var producer = new StorageManagerProducer(root); + var iterable = edges(producer, query("FROM FROM FROM ")); + var iterator = iterable.iterator(); + try { + assertNotNull(seen(iterator)); var actual = rows(iterator); assertEquals(oracle, actual); + for (int i = 0; i < oracle.size(); i++) assertSame(oracle.get(i), actual.get(i)); + } finally { producer.closeEdges(iterable); producer.close(); } + } + } + + @Test void writerAddingAPartitionBetweenFindAndMetadataInspectionKeepsTheSingleSnapshot() throws Exception { + try (var root = storage()) { + root.mutations().add(row(0, G)); + var wrapper = observed(root, pattern -> { + var stream = root.queries().find(pattern); + root.mutations().add(row(0, H)); return stream; + }); + var producer = new StorageManagerProducer(wrapper); + var iterable = edges(producer, query("FROM FROM ")); + var iterator = iterable.iterator(); + try { assertNull(seen(iterator)); assertEquals(List.of(row(0, G)), rows(iterator)); } + finally { producer.closeEdges(iterable); producer.close(); } + } + } + + @Test void writerDroppingAPartitionBetweenFindAndMetadataInspectionStillDeduplicatesTheOldUnion() throws Exception { + try (var root = storage()) { + root.mutations().add(row(0, G)); root.mutations().add(row(0, H)); + var wrapper = observed(root, pattern -> { + var stream = root.queries().find(pattern); + root.mutations().dropGraph(H); return stream; + }); + var producer = new StorageManagerProducer(wrapper); + var iterable = edges(producer, query("FROM FROM ")); + var iterator = iterable.iterator(); + try { assertNotNull(seen(iterator)); assertEquals(List.of(row(0, G)), rows(iterator)); } + finally { producer.closeEdges(iterable); producer.close(); } + } + } + + @Test void unknownWrappedTraversalConservativelyDeduplicatesMergedContexts() throws Exception { + try (var root = storage()) { + root.mutations().add(row(0, G)); root.mutations().add(row(0, H)); + var wrapper = observed(root, pattern -> root.queries().find(pattern).peek(s -> { })); + var producer = new StorageManagerProducer(wrapper); + var iterable = edges(producer, query("FROM FROM ")); + var iterator = iterable.iterator(); + try { assertNotNull(seen(iterator)); assertEquals(List.of(row(0, G)), rows(iterator)); } + finally { producer.closeEdges(iterable); producer.close(); } + } + } + + @Test void sparqlProjectionBagsAndExplicitNamedGraphsKeepTheirMultiplicity() { + try (var root = storage()) { + root.mutations().add(row(0, G)); root.mutations().add(row(0, H)); + root.mutations().add(Values.factory().createStatement(new SimpleIRI("urn:s0"), new SimpleIRI("urn:p2"), new SimpleIRI("urn:o"), G)); + var executor = new NextSparqlPipelineExecutor(root); + try (var result = executor.evaluateTuple("SELECT ?s FROM FROM { ?s ?p ?o }")) { + assertEquals(List.of("urn:s0", "urn:s0"), result.stream().map(b -> b.getValue("s").stringValue()).toList()); + } + try (var result = executor.evaluateTuple("SELECT ?s FROM { ?s ?p ?o }")) { assertEquals(2, result.stream().count()); } + try (var result = executor.evaluateTuple("SELECT ?s FROM NAMED FROM NAMED { GRAPH ?g { ?s ?o } }")) { + assertEquals(2, result.stream().count()); + } + } + } +} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/StorageBenchmark.java b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/StorageBenchmark.java new file mode 100644 index 000000000..d577a0dac --- /dev/null +++ b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/StorageBenchmark.java @@ -0,0 +1,70 @@ +package fr.inria.corese.core.next.storage.impl.memory; + +import fr.inria.corese.core.next.data.Values; +import fr.inria.corese.core.next.data.api.model.Statement; +import fr.inria.corese.core.next.data.api.term.IRI; +import fr.inria.corese.core.next.data.api.term.Resource; +import fr.inria.corese.core.next.data.api.term.SimpleIRI; +import fr.inria.corese.core.next.data.api.term.SimpleLiteral; +import fr.inria.corese.core.next.data.api.term.Value; + +import java.util.Iterator; +import java.util.NoSuchElementException; + +/** + * Test dataset generator used across storage unit and integration tests. + */ +public final class StorageBenchmark { + + private static final IRI[] PROPERTIES = { + iri("type"), iri("group"), iri("name"), iri("knows"), iri("score"), iri("tag") + }; + + private StorageBenchmark() { + // Utility class + } + + private static IRI iri(String local) { + return new SimpleIRI("urn:bench:" + local); + } + + /** + * Generates a deterministic synthetic dataset of statements. + * + * @param people number of entities (6 statements per entity) + * @param graphs number of named graphs (0 for default graph) + * @param skew whether to introduce skew in friend relations + * @return an iterable providing the statements on demand + */ + public static Iterable data(int people, int graphs, boolean skew) { + return () -> new Iterator<>() { + int position; + + @Override + public boolean hasNext() { + return position < people * 6; + } + + @Override + public Statement next() { + if (!hasNext()) { + throw new NoSuchElementException(); + } + int person = position / 6; + int slot = position++ % 6; + Resource subject = iri("s" + person); + int friend = skew && person % 5 != 0 ? person % Math.max(1, people / 100) : (person + 1) % people; + Value object = switch (slot) { + case 0 -> iri("Person"); + case 1 -> iri("g" + person % 100); + case 2 -> new SimpleLiteral("name" + person); + case 3 -> iri("s" + friend); + case 4 -> new SimpleLiteral(Integer.toString(person % 100), new SimpleIRI("http://www.w3.org/2001/XMLSchema#integer")); + default -> iri("tag" + person % 32); + }; + Resource context = graphs == 0 || person % 100 < 50 ? null : iri("graph" + person % graphs); + return Values.factory().createStatement(subject, PROPERTIES[slot], object, context); + } + }; + } +} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/StorageIndexShapeBenchmark.java b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/StorageIndexShapeBenchmark.java new file mode 100644 index 000000000..01f88ab63 --- /dev/null +++ b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/StorageIndexShapeBenchmark.java @@ -0,0 +1,113 @@ +package fr.inria.corese.core.next.storage.impl.memory; + +import fr.inria.corese.core.next.data.Values; +import fr.inria.corese.core.next.data.api.model.Statement; +import fr.inria.corese.core.next.data.api.term.IRI; +import fr.inria.corese.core.next.data.api.term.Resource; +import fr.inria.corese.core.next.data.api.term.SimpleIRI; +import fr.inria.corese.core.next.storage.Storages; +import fr.inria.corese.core.next.storage.api.config.StorageConfig; +import fr.inria.corese.core.next.storage.api.model.StatementPattern; + +import java.lang.management.ManagementFactory; +import java.util.ArrayList; +import java.util.HashSet; +import java.util.List; +import java.util.function.LongSupplier; + +/** Adversarial index layouts: all-dense prefixes and triples with no shared terms. */ +@SuppressWarnings({"java:S106", "java:S1215", "java:S1659", "java:S3078", "java:S3776", "java:S3973"}) +public final class StorageIndexShapeBenchmark { + private static volatile long consumed; + private static IRI iri(String name) { return new SimpleIRI("urn:shape:" + name); } + private static long heap() { return ManagementFactory.getMemoryMXBean().getHeapMemoryUsage().getUsed(); } + private static void collect() { System.gc(); System.gc(); } + private static long gcCount() { + return ManagementFactory.getGarbageCollectorMXBeans().stream().mapToLong(b -> Math.max(0, b.getCollectionCount())).sum(); + } + private static long gcMillis() { + return ManagementFactory.getGarbageCollectorMXBeans().stream().mapToLong(b -> Math.max(0, b.getCollectionTime())).sum(); + } + private static void measure(String operation, int iteration, LongSupplier action) { + var bean = (com.sun.management.ThreadMXBean) ManagementFactory.getThreadMXBean(); + if (!bean.isThreadAllocatedMemoryEnabled()) bean.setThreadAllocatedMemoryEnabled(true); + long allocated = bean.getThreadAllocatedBytes(Thread.currentThread().threadId()), gc = gcCount(), gcTime = gcMillis(), start = System.nanoTime(); + long result = action.getAsLong(); + long nanos = System.nanoTime() - start; + consumed ^= result; + System.out.printf(java.util.Locale.ROOT, "operation,%s,%d,%.6f,%d,%d,%d,%d%n", operation, iteration, + nanos / 1e6, bean.getThreadAllocatedBytes(Thread.currentThread().threadId()) - allocated, + gcCount() - gc, gcMillis() - gcTime, result); + } + + public static void main(String[] arguments) { + String backend = arguments[0], mode = arguments[1], shape = arguments[2]; + int target = Integer.parseInt(arguments[3]), warmups = Integer.parseInt(arguments[4]), repeats = Integer.parseInt(arguments[5]); + List input = new ArrayList<>(); + if (shape.equals("dense")) { + IRI[] predicates = new IRI[32], objects = new IRI[32]; + for (int i = 0; i < 32; i++) { predicates[i] = iri("p" + i); objects[i] = iri("o" + i); } + for (int s = 0; s < target / 1024; s++) { + IRI subject = iri("s" + s); + for (IRI predicate : predicates) for (IRI object : objects) + input.add(Values.factory().createStatement(subject, predicate, object)); + } + } else if (shape.equals("unique")) { + for (int i = 0; i < target; i++) + input.add(Values.factory().createStatement(iri("s" + i), iri("p" + i), iri("o" + i))); + } else throw new IllegalArgumentException("shape"); + if (input.isEmpty()) throw new IllegalArgumentException("Dense target must be at least 1024"); + System.out.println("environment," + backend + ',' + mode + ',' + shape + ',' + input.size()); + for (int iteration = -warmups; iteration < repeats; iteration++) { + try (var storage = Storages.create(StorageConfig.builder().type(backend).build())) { + collect(); long baseline = heap(); + measure("load", iteration, () -> { + long added = mode.equals("batch") ? storage.mutations().addAll(input) + : input.stream().filter(storage.mutations()::add).count(); + if (added != input.size()) throw new AssertionError("load"); + return added; + }); + collect(); + System.out.println("heap,load," + iteration + ',' + (heap() - baseline) + ",0"); + Statement first = input.getFirst(); + for (int mask = 0; mask < 8; mask++) { + var pattern = StatementPattern.of((mask & 1) == 0 ? null : first.getSubject(), + (mask & 2) == 0 ? null : first.getPredicate(), (mask & 4) == 0 ? null : first.getObject(), (Resource) null); + var expected = new HashSet(); + for (Statement row : input) { + if ((pattern.getSubject() == null || pattern.getSubject().equals(row.getSubject())) + && (pattern.getPredicate() == null || pattern.getPredicate().equals(row.getPredicate())) + && (pattern.getObject() == null || pattern.getObject().equals(row.getObject()))) expected.add(row); + } + measure("count-mask-" + mask, iteration, () -> { + long count = storage.queries().count(pattern); + if (count != expected.size()) throw new AssertionError("count"); + return count; + }); + measure("find-mask-" + mask, iteration, () -> { + try (var rows = storage.queries().find(pattern)) { + long count = rows.count(); + if (count != expected.size()) throw new AssertionError("find count"); + return count; + } + }); + try (var rows = storage.queries().find(pattern)) { + List actual = rows.toList(); + if (actual.size() != expected.size() || !new HashSet<>(actual).equals(expected)) + throw new AssertionError("full statement oracle"); + } + System.out.println("verified,find-mask-" + mask + ',' + iteration); + } + measure("remove-reinsert-128", iteration, () -> { + for (Statement row : input.subList(0, Math.min(128, input.size()))) { + if (!storage.mutations().remove(row) || !storage.mutations().add(row)) throw new AssertionError("mutation"); + } + return 128; + }); + if (storage.queries().count(StatementPattern.matchAll()) != input.size()) throw new AssertionError("final size"); + if (storage.queries() instanceof IndexedStatementStore indexed) MemoryStorageAssertions.verify(indexed); + } + } + System.out.println("consumed," + consumed); + } +} diff --git a/src/test/java/fr/inria/corese/core/next/storage/impl/memory/TermKeyTest.java b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/TermKeyTest.java new file mode 100644 index 000000000..b0e3ff76d --- /dev/null +++ b/src/test/java/fr/inria/corese/core/next/storage/impl/memory/TermKeyTest.java @@ -0,0 +1,63 @@ +package fr.inria.corese.core.next.storage.impl.memory; + +import fr.inria.corese.core.next.data.api.term.SimpleIRI; +import fr.inria.corese.core.next.data.api.term.SimpleLiteral; +import org.junit.jupiter.api.Test; + +import java.util.HashMap; +import java.util.Random; + +import static org.junit.jupiter.api.Assertions.*; + +class TermKeyTest { + @Test void nativeIRIHashMatchesFullStringAcrossSplitsUnicodeAndLongNames() { + Random random = new Random(813719); + for (int length : new int[]{0, 1, 7, 31, 32, 33, 63, 256, 2000}) { + StringBuilder local = new StringBuilder(); + for (int i = 0; i < length; i++) local.append((char) ('a' + random.nextInt(26))); + // A fragment avoids the unrelated namespace parser's long-path regex. + String full = "urn:hash:#" + local; + for (int split : new int[]{4, 10, full.length() - 1, full.length()}) { + var iri = new SimpleIRI(full.substring(0, split), full.substring(split)); + assertEquals(full.hashCode(), new TermKey(iri).hashCode(), "length=" + length + ", split=" + split); + } + } + String unicode = "urn:hash:#é😀"; + for (int split : new int[]{4, 10, unicode.length() - 1, unicode.length()}) { + var iri = new SimpleIRI(unicode.substring(0, split), unicode.substring(split)); + assertEquals(unicode.hashCode(), new TermKey(iri).hashCode()); + } + } + + @Test void equalIRIsWithDifferentViewsRemainLookupCompatibleAndCollisionsStayDistinct() { + var first = new TermKey(new SimpleIRI("urn:", "view:item")); + var second = new TermKey(new SimpleIRI("urn:view:", "item")); + assertEquals(first, second); + assertEquals(second, first); + assertEquals(first.hashCode(), second.hashCode()); + HashMap map = new HashMap<>(); + map.put(first, "same IRI"); + assertEquals("same IRI", map.get(second)); + var collisionA = new TermKey(new SimpleIRI("urn:", "Aa")); + var collisionB = new TermKey(new SimpleIRI("urn:", "BB")); + assertEquals(collisionA.hashCode(), collisionB.hashCode()); + assertNotEquals(collisionA, collisionB); + map.put(collisionA, "A"); + map.put(collisionB, "B"); + assertEquals("A", map.get(new TermKey(new SimpleIRI("urn:Aa")))); + assertEquals("B", map.get(new TermKey(new SimpleIRI("urn:BB")))); + } + + @Test void literalDatatypeViewsShareKeysWhileLexicalFormsRemainDistinct() { + var datatypeA = new SimpleIRI("urn:", "datatype:item"); + var datatypeB = new SimpleIRI("urn:datatype:", "item"); + var first = new TermKey(new SimpleLiteral("01", datatypeA)); + var second = new TermKey(new SimpleLiteral("01", datatypeB)); + assertEquals(first, second); + assertEquals(first.hashCode(), second.hashCode()); + int expected = 31 * (31 * "01".hashCode() + datatypeA.stringValue().hashCode()) + + java.util.Optional.empty().hashCode(); + assertEquals(expected, first.hashCode()); + assertNotEquals(first, new TermKey(new SimpleLiteral("1", datatypeA))); + } +}