Milvus: add the MEVD provider by Overvolted · Pull Request #77 · CommunityToolkit/AI

Overvolted · 2026-10-23T17:42:20Z

Adds CommunityToolkit.VectorData.Milvus, the Milvus provider proposed in #76, with MEVD/test/Milvus.UnitTests and MEVD/test/Milvus.ConformanceTests. It follows the layout of the Qdrant provider and builds on Milvus.Client 2.6.0. It needs Milvus 2.6.

The conformance tests start milvusdb/milvus:v2.6.25 with Testcontainers.Milvus, so the mevd.yml matrix picks them up. Every shared suite runs. The overrides assert NotSupportedException for what Milvus does not have: ordering, models without vectors, and the distance functions other than cosine, dot product, Euclidean and squared Euclidean. The conformance project targets .NET 10 only: on .NET Framework, Milvus.Client needs TLS, and the test container uses plaintext gRPC. The README says what .NET Framework needs.

The tests of full-text indexing and hybrid search mirror the collection definitions of the Semantic Kernel TextSearchStore and of the Agent Framework ChatHistoryMemoryProvider, which mark their text as full-text indexed.

Co-authored by @Laurianti

Related: microsoft/semantic-kernel#12509, microsoft/semantic-kernel#13391, milvus-io/milvus-sdk-csharp#102.

Fixes #76

Add the Milvus.Client-backed package, typed and dynamic collections, scalar and array filtering, vector and BM25 hybrid search, dependency injection, documentation, and unit and conformance tests. Co-authored-by: Federico Laurianti <laurianti@gmail.com>

Overvolted · 2026-10-23T17:43:49Z

@dotnet-policy-service agree

Copilot

🟡 Changes recommended

Initialization recovery and reusable test-container lifetime issues remain unresolved.

2 open issues

What changed in this PR

Adds CommunityToolkit.VectorData.Milvus to the toolkit’s Microsoft.Extensions.VectorData providers, addressing #76 with Milvus 2.6 support.

Changes:

Implements typed and dynamic collections, filtering, vector search, BM25 hybrid search, and dependency injection.
Adds unit tests and container-backed conformance suites, including explicit checks for unsupported capabilities.
Registers the projects and dependencies and documents setup and platform limitations.

File	Description
README.md	Adds Milvus package links and badges.
MEVD/test/Milvus.UnitTests/MilvusTestRecord.cs	Defines a shared test model.
MEVD/test/Milvus.UnitTests/MilvusStringEscapingTests.cs	Tests Unicode and string escaping.
MEVD/test/Milvus.UnitTests/MilvusMappingTests.cs	Tests schemas, mapping, metrics, and validation.
MEVD/test/Milvus.UnitTests/MilvusHybridSearchTests.cs	Tests hybrid-search registration and validation.
MEVD/test/Milvus.UnitTests/MilvusFullTextTests.cs	Tests BM25 schemas and consumer definitions.
MEVD/test/Milvus.UnitTests/MilvusFilterTranslatorTests.cs	Tests scalar filters and null semantics.
MEVD/test/Milvus.UnitTests/MilvusEnsureCollectionTests.cs	Tests collection initialization and load timeouts.
MEVD/test/Milvus.UnitTests/MilvusDatabaseTests.cs	Tests database metadata and authentication options.
MEVD/test/Milvus.UnitTests/MilvusCollectionTests.cs	Tests collection services and input validation.
MEVD/test/Milvus.UnitTests/MilvusClientLifetimeTests.cs	Tests client ownership and error wrapping.
MEVD/test/Milvus.UnitTests/MilvusArrayTests.cs	Tests array schemas and mapping.
MEVD/test/Milvus.UnitTests/MilvusArrayFilterTests.cs	Tests array-membership translation.
MEVD/test/Milvus.UnitTests/Milvus.UnitTests.csproj	Configures the unit-test project.
MEVD/test/Milvus.UnitTests/.editorconfig	Configures test-specific diagnostics.
MEVD/test/Milvus.ConformanceTests/TypeTests/MilvusKeyTypeTests.cs	Exercises supported key types.
MEVD/test/Milvus.ConformanceTests/TypeTests/MilvusEmbeddingTypeTests.cs	Enables embedding-type conformance tests.
MEVD/test/Milvus.ConformanceTests/TypeTests/MilvusDataTypeTests.cs	Defines supported data-type coverage.
MEVD/test/Milvus.ConformanceTests/Support/MilvusTestStore.cs	Manages the test server and clients.
MEVD/test/Milvus.ConformanceTests/Support/MilvusFixture.cs	Provides the shared store fixture.
MEVD/test/Milvus.ConformanceTests/ModelTests/MilvusNoVectorModelTests.cs	Verifies rejection of vectorless models.
MEVD/test/Milvus.ConformanceTests/ModelTests/MilvusNoDataModelTests.cs	Enables models without data fields.
MEVD/test/Milvus.ConformanceTests/ModelTests/MilvusMultiVectorModelTests.cs	Enables multiple-vector model coverage.
MEVD/test/Milvus.ConformanceTests/ModelTests/MilvusDynamicModelTests.cs	Covers dynamic models and ordering rejection.
MEVD/test/Milvus.ConformanceTests/ModelTests/MilvusBasicModelTests.cs	Covers typed models and ordering rejection.
MEVD/test/Milvus.ConformanceTests/MilvusTestSuiteImplementationTests.cs	Checks shared-suite implementation.
MEVD/test/Milvus.ConformanceTests/MilvusRoundTripTests.cs	Exercises storage, retrieval, and search.
MEVD/test/Milvus.ConformanceTests/MilvusNullComparisonTests.cs	Checks filter null semantics against Milvus.
MEVD/test/Milvus.ConformanceTests/MilvusIndexKindTests.cs	Enables index-kind conformance tests.
MEVD/test/Milvus.ConformanceTests/MilvusHybridSearchTests.cs	Exercises hybrid ranking and search options.
MEVD/test/Milvus.ConformanceTests/MilvusFullTextTests.cs	Exercises consumer-style full-text models.
MEVD/test/Milvus.ConformanceTests/MilvusFilterTests.cs	Enables shared filter coverage.
MEVD/test/Milvus.ConformanceTests/MilvusEnsureCollectionTests.cs	Tests preservation of existing collections.
MEVD/test/Milvus.ConformanceTests/MilvusEmbeddingGenerationTests.cs	Covers embedding generation and registration.
MEVD/test/Milvus.ConformanceTests/MilvusDistanceFunctionTests.cs	Covers supported and rejected metrics.
MEVD/test/Milvus.ConformanceTests/MilvusDependencyInjectionTests.cs	Exercises registration overloads and lifetimes.
MEVD/test/Milvus.ConformanceTests/MilvusDatabaseTests.cs	Verifies selected-database isolation.
MEVD/test/Milvus.ConformanceTests/MilvusCollectionManagementTests.cs	Enables collection-management coverage.
MEVD/test/Milvus.ConformanceTests/MilvusArrayFilterTests.cs	Exercises array filters against Milvus.
MEVD/test/Milvus.ConformanceTests/Milvus.ConformanceTests.csproj	Configures .NET 10 container-backed tests.
MEVD/src/Milvus/SharedMilvusClient.cs	Shares client ownership through reference counting.
MEVD/src/Milvus/README.md	Documents setup, capabilities, and limitations.
MEVD/src/Milvus/MilvusVectorStoreOptions.cs	Configures default embedding generation.
MEVD/src/Milvus/MilvusVectorStore.cs	Implements store-level collection access.
MEVD/src/Milvus/MilvusServiceCollectionExtensions.cs	Adds keyed and unkeyed registrations.
MEVD/src/Milvus/MilvusModelBuilder.cs	Validates models and storage names.
MEVD/src/Milvus/MilvusMapper.cs	Maps records and vectors to storage.
MEVD/src/Milvus/MilvusFilterTranslator.cs	Translates expressions into Milvus filters.
MEVD/src/Milvus/MilvusFieldMapping.cs	Maps field types, columns, and literals.
MEVD/src/Milvus/MilvusErrorHandler.cs	Wraps SDK failures with operation metadata.
MEVD/src/Milvus/MilvusDynamicCollection.cs	Supports dictionary-based collections.
MEVD/src/Milvus/MilvusConstants.cs	Defines the provider identifier.
MEVD/src/Milvus/MilvusCollectionSearchMapping.cs	Maps scores and threshold direction.
MEVD/src/Milvus/MilvusCollectionOptions.cs	Defines collection configuration.
MEVD/src/Milvus/MilvusCollectionCreateMapping.cs	Builds schemas and BM25 functions.
MEVD/src/Milvus/MilvusCollection.cs	Implements collection operations and searches.
MEVD/src/Milvus/Milvus.csproj	Defines the provider package and target frameworks.
MEVD/src/Milvus/AssemblyInfo.cs	Adds the assembly-info file with license headers.
MEVD/MEVD.slnf	Includes the provider and test projects.
Directory.Packages.props	Adds Milvus SDK and Testcontainers versions.
CommunityToolkit.AI.slnx	Registers the three Milvus projects.

_{Review effort: Balanced}

adamsitnik · 2026-10-27T09:44:19Z

+        {
+            // The indexes are created with the collection. Those of an existing collection stay as they are: Milvus rejects
+            // an index with other settings on an indexed field, or on a field the collection does not have.
+            if (!await _milvusClient.Client.HasCollectionAsync(Name, cancellationToken: cancellationToken).ConfigureAwait(false))


Out other existing providers also don't handle index creation failure really well. I think it's acceptable to keep it as implemented here, or just delete the collection on failed index creation and re-throw the index creation exception.

Thanks, I’ll keep the current behavior. I’d prefer to avoid automatically deleting collections here to eliminate any risk of deleting user data.

adamsitnik

@Overvolted big thanks for your contribution @Overvolted !

The contribution is very promising, I found some issues, but most of them are just subjective nits. PTAL at my comments

adamsitnik · 2026-10-27T09:07:07Z

+    /// <inheritdoc />
+    protected override void Dispose(bool disposing)
+    {
+        if (Interlocked.Exchange(ref _disposed, 1) == 0)


Do we really need to store this information in an extra field of SharedMilvusClient has it's own reference count?

Hi @adamsitnik ! Thanks for your review, the reason why i did this is that if MulvusCollection.Dispose() is called multiple times by mistake, it would skew the ClientWrapper internal counter causing it to raise ObjectDisposedExceptions to other consumers, I will add some tests to demostrate this behavior.

Added 2 tests in 2c6223a

let me know if you want to drop this check anyway

adamsitnik

@Overvolted thank you for addressing my feedback! Here is the next round (very likely the final round).

adamsitnik · 2026-10-28T09:59:53Z

+
+    /// <summary>Whether the collection was disposed: it releases its share of the client only once.</summary>
+    private int _disposed;


Suggested change

/// <summary>Whether the collection was disposed: it releases its share of the client only once.</summary>

private int _disposed;

adamsitnik · 2026-10-28T10:01:10Z

+        if (Interlocked.Exchange(ref _disposed, 1) == 0)
+        {
+            _clientWrapper.Dispose();
+        }


Suggested change

if (Interlocked.Exchange(ref _disposed, 1) == 0)

{

_clientWrapper.Dispose();

}

_clientWrapper.Dispose();

adamsitnik · 2026-10-28T10:29:48Z

+  <PropertyGroup>
+    <AssemblyName>CommunityToolkit.VectorData.Milvus.UnitTests</AssemblyName>
+    <RootNamespace>CommunityToolkit.VectorData.Milvus.UnitTests</RootNamespace>
+    <TargetFramework>net10.0</TargetFramework>


Is .NET Framework support missing here on purpose?

Hi @adamsitnik, this was inherited from Qdrant’s unit-test project, (which also targets only net10.0).

Adding net472 requires adapting the mock HTTP helpers because .NET Framework has limited support for gRPC over HTTP/2 (https://learn.microsoft.com/en-us/aspnet/core/grpc/netstandard?view=aspnetcore-10.0#net-framework).

I'm thinking that maybe having some kind of coverage on net472 would be nice since we cannot do it in the conformance tests.. I can try to work around it if you want,

Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>

…olted/AI into feature/milvus-vector-data

adamsitnik

LGTM, big thanks for your great contribution @Overvolted !

adamsitnik · 2026-10-29T11:20:10Z

I've started the procedure of publishing the package to nuget.org. When it's done for the first time I need to add .NET Foundation as an owner, and I typically need to wait for somebody from US to wake up, check the email and approve.

adamsitnik · 2026-10-29T17:02:52Z

It's live!! https://www.nuget.org/packages/CommunityToolkit.VectorData.Milvus/

Overvolted · 2026-10-29T21:20:32Z

Let's go! Thank you very much @adamsitnik for your reviews!

### Motivation and Context Follow the Chroma cleanup in microsoft#14579 now that the Milvus provider has merged in CommunityToolkit/AI#77 and is published as [CommunityToolkit.VectorData.Milvus](https://www.nuget.org/packages/CommunityToolkit.VectorData.Milvus). ### Description - Remove the legacy .NET Milvus memory store and its tests. - Replace its README with a deprecation notice linking to the Toolkit package and implementation. - Remove unused Milvus package references and the experimental-feature entry. - Delete `MEVD.slnx`, which has no projects left after this removal. Validation: - `git diff --check` — passed. - `dotnet msbuild dotnet/src/IntegrationTests/IntegrationTests.csproj -getItem:PackageReference -verbosity:quiet` — passed; evaluated package references contain no Milvus dependency. - `dotnet msbuild dotnet/src/Experimental/Process.IntegrationTestRunner.Dapr/Process.IntegrationTestRunner.Dapr.csproj -getItem:PackageReference -verbosity:quiet` — passed; evaluated package references contain no Milvus dependency. - Full builds, unit tests, and `dotnet format` were not run. The Milvus implementation and test project are removed; the remaining changes are documentation, solution, and dependency cleanup. ### Contribution Checklist - [ ] The code builds clean without any errors or warnings - [ ] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [ ] All unit tests pass, and I have added new tests where possible - [ ] I didn't break anyone 😄 Co-authored-by: adamsitnik <6011991+adamsitnik@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

adamsitnik · 2026-10-30T17:32:07Z

The docs got updated in MicrosoftDocs/semantic-kernel-docs#460 and are now live:

https://learn.microsoft.com/en-us/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/milvus-connector

The Semantic Kernel provider got removed: microsoft/semantic-kernel#14682

and the SK NuGet package got deprecated and points to the new package now:

https://www.nuget.org/packages/Microsoft.SemanticKernel.Connectors.Milvus/

Add Milvus vector data provider

78e5b1f

Add the Milvus.Client-backed package, typed and dynamic collections, scalar and array filtering, vector and BM25 hybrid search, dependency injection, documentation, and unit and conformance tests. Co-authored-by: Federico Laurianti <laurianti@gmail.com>

Laurianti requested a review from adamsitnik October 23, 2026 17:47

adamsitnik requested a balanced review from Copilot October 27, 2026 08:21

Copilot started reviewing on behalf of adamsitnik October 27, 2026 08:22

View session

Copilot AI reviewed Oct 27, 2026

View reviewed changes

adamsitnik reviewed Oct 27, 2026

View reviewed changes

Overvolted added 8 commits October 27, 2026 21:45

style: standardize member ordering across Milvus classes

943a871

style: remove redundant Milvus constructor comments

f5961ca

docs: explain Milvus HNSW index parameters

cb9ba98

refactor: align Milvus client wrapper naming with provider conventions

097bab1

perf: avoid list allocation for single-record Milvus lookups

8336736

refactor: centralize Milvus key type validation

e749425

refactor: centralize Milvus limits and default values

56a3b1d

style: normalize namespace spacing in Milvus files

bbf4f47

adamsitnik reviewed Oct 28, 2026

View reviewed changes

Overvolted added 3 commits October 28, 2026 21:13

test: cover idempotent disposal of shared Milvus clients

2c6223a

feat: expose Milvus collection load timeout in options

e76300a

feat: make Milvus HNSW index parameters configurable

b63a8a9

Overvolted mentioned this pull request Oct 28, 2026

Search with no matching records throws when output fields are requested milvus-io/milvus-sdk-csharp#191

Open

Overvolted and others added 10 commits October 28, 2026 21:35

docs: link Milvus empty-search workaround to upstream issue

265b2bf

style: Add newlines between methods

1c7a722

Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>

style: Add newline after class

0ed6811

Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>

refactor: separate Milvus authentication registration overloads

3a954d3

fix: default Milvus host registrations to HTTPS

a2afcca

style: break long line

3d22cd0

Co-authored-by: Adam Sitnik <adam.sitnik@gmail.com>

docs: avoid root username in Milvus example

58fe415

Merge branch 'feature/milvus-vector-data' of https://github.com/Overv…

37b5943

…olted/AI into feature/milvus-vector-data

style: normalize member spacing in Milvus classes

5c30e71

style: remove trailing blank line in Milvus string escaping tests

aeab801

Overvolted added 2 commits October 28, 2026 22:19

fix: reject minimum_should_match as a Milvus field name

ccf84f0

fix: preserve reusable Milvus test container between fixtures

f55166c

adamsitnik approved these changes Oct 29, 2026

View reviewed changes

adamsitnik merged commit e0a1ec3 into CommunityToolkit:main Oct 29, 2026
18 checks passed

adamsitnik mentioned this pull request Oct 29, 2026

.Net: Remove migrated .NET Milvus connector microsoft/semantic-kernel#14682

Merged

4 tasks

Overvolted mentioned this pull request Oct 29, 2026

Support for Microsoft.Extensions.VectorData milvus-io/milvus-sdk-csharp#102

Open


		/// <summary>Whether the collection was disposed: it releases its share of the client only once.</summary>
		private int _disposed;

Conversation

Overvolted commented Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Overvolted commented Oct 23, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

adamsitnik left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

adamsitnik left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

adamsitnik left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

adamsitnik commented Oct 29, 2026

Uh oh!

adamsitnik commented Oct 29, 2026

Uh oh!

Overvolted commented Oct 29, 2026

Uh oh!

adamsitnik commented Oct 30, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Overvolted commented Oct 23, 2026 •

edited

Loading