Skip to content

Add cross-property BM25 And operator - #2114

Merged
amourao merged 3 commits into
dev/1.39from
worktree-bm25-search-mode
Aug 3, 2026
Merged

Add cross-property BM25 And operator#2114
amourao merged 3 commits into
dev/1.39from
worktree-bm25-search-mode

Conversation

@amourao

@amourao amourao commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Exposes the OPERATOR_AND_CROSS search operator (Weaviate 1.39.0, backported to 1.37.15 and 1.38.8) as BM25Operator.and_cross(), available wherever a BM25 operator is accepted: query.bm25, query.hybrid, the generative-query equivalents, and aggregate.hybrid.

Unlike and_(), which requires every query token to occur within a single property, and_cross() lets a token be matched by any of the searched properties as long as each token is matched by at least one. All searched properties must share the same tokenization and analyzer settings — the server rejects the query otherwise.

collection.query.bm25("banana split", operator=wvc.query.BM25Operator.and_cross())

Changes

  • BM25OperatorAndCross + BM25Operator.and_cross() in weaviate/collections/classes/grpc.py.
  • The duplicated SearchOperatorOptions construction in the bm25 and hybrid paths is folded into _BaseGRPC._bm25_operator_to_grpc(), so the version gate covers every caller. Using and_cross() against an older server raises WeaviateUnsupportedFeatureError rather than sending an enum value the server does not understand.
  • _ServerVersion.is_at_least_any() expresses a minimum that was backported across several release branches (a 1.38.x server needs 1.38.8, a 1.37.x server needs 1.37.15).
  • Regenerated protos for the OPERATOR_AND_CROSS enum value.
  • CI's 1.39 image bumped to 1.39.0-rc.1-89299a5 — the first RC carrying the operator.

Tests

  • test/collection/test_bm25_operator.pyand_cross() maps to OPERATOR_AND_CROSS on the bm25 and hybrid requests, and is rejected on servers that predate it.
  • test/test_server_version.pyis_at_least_any() around each branch boundary.
  • integration/test_collection.py — one cross-property match against a live server, version-gated with a skip.

Exposes the OPERATOR_AND_CROSS search operator added in Weaviate 1.39.0
(backported to 1.37.15 and 1.38.8) as `BM25Operator.and_cross()`. Unlike
`and_()`, a query token may be matched by any of the searched properties
instead of having to occur within a single one.

The gRPC construction for the bm25 and hybrid paths is folded into a shared
`_BaseGRPC._bm25_operator_to_grpc()` so the version gate applies to query,
generative-query and aggregation alike; sending the new enum to a server that
predates it raises WeaviateUnsupportedFeatureError.

`_ServerVersion.is_at_least_any()` is added to express minimums that were
backported across several release branches.
@amourao
amourao requested a review from a team as a code owner July 31, 2026 14:59

@orca-security-eu orca-security-eu Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orca Security Scan Summary

Status Check Issues by priority
Passed Passed Secrets high 0   medium 0   low 0   info 0 View in Orca

@codecov-commenter

codecov-commenter commented Jul 31, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.00000% with 1 line in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (dev/1.39@ae327ca). Learn more about missing BASE report.

Files with missing lines Patch % Lines
integration/test_collection.py 90.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             dev/1.39    #2114   +/-   ##
===========================================
  Coverage            ?   88.34%           
===========================================
  Files               ?      303           
  Lines               ?    23323           
  Branches            ?        0           
===========================================
  Hits                ?    20605           
  Misses              ?     2718           
  Partials            ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread integration/test_collection.py Outdated
Comment thread test/test_server_version.py
Comment thread test/collection/test_bm25_operator.py Outdated
Comment thread integration/test_collection.py Outdated
Comment thread docs/changelog.rst Outdated
- Changelog is written at version creation, so the entry is removed.
- Mismatched-tokenization integration test dropped: the server validates that.
- Unsupported-version integration tests dropped for bm25 and hybrid: the
  version gate is already covered by unit tests.
- Hybrid cross-property integration test dropped: it exercises the same server
  path as the bm25 one, and the request wiring is unit tested.
- Unit tests cut to the two behaviours that are actually new: and_cross maps to
  OPERATOR_AND_CROSS on the bm25 and hybrid requests, and older servers are
  rejected.
@dirkkul

dirkkul commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

BM25OperatorAndCross (weaviate/collections/classes/grpc.py:676) isn't exported from weaviate/outputs/query.py, but its two siblings are: BM25OperatorAnd and BM25OperatorOr appear both in the import block (outputs/query.py:11-12) and in __all__ (:64-65).

That leaves the public surface asymmetric for no reason — users who want to isinstance-check or type-annotate against the new operator can't import it from the public namespace the way they can for the other two. One line in each spot:

from weaviate.collections.classes.grpc import (
    MMR,
    BM25OperatorAnd,
    BM25OperatorAndCross,
    BM25OperatorOr,
    ...
)

__all__ = [
    ...
    "BM25OperatorAnd",
    "BM25OperatorAndCross",
    "BM25OperatorOr",
    ...
]

Its two siblings, BM25OperatorAnd and BM25OperatorOr, are already exported, so
isinstance checks and type annotations against the new operator have to reach
into the private module without this.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants