AICentral 0.5.3-pullrequest0025-0047

This is a prerelease version of AICentral.
There is a newer version of this package available.
See the version list below for details.
dotnet add package AICentral --version 0.5.3-pullrequest0025-0047
                    
NuGet\Install-Package AICentral -Version 0.5.3-pullrequest0025-0047
                    
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="AICentral" Version="0.5.3-pullrequest0025-0047" />
                    
For projects that support PackageReference, copy this XML node into the project file to reference the package.
<PackageVersion Include="AICentral" Version="0.5.3-pullrequest0025-0047" />
                    
Directory.Packages.props
<PackageReference Include="AICentral" />
                    
Project file
For projects that support Central Package Management (CPM), copy this XML node into the solution Directory.Packages.props file to version the package.
paket add AICentral --version 0.5.3-pullrequest0025-0047
                    
#r "nuget: AICentral, 0.5.3-pullrequest0025-0047"
                    
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
#addin nuget:?package=AICentral&version=0.5.3-pullrequest0025-0047&prerelease
                    
Install AICentral as a Cake Addin
#tool nuget:?package=AICentral&version=0.5.3-pullrequest0025-0047&prerelease
                    
Install AICentral as a Cake Tool

AI Central

AI Central gives you control over your AI services.

  • Intelligent Routing
  • Custom consumer OAuth2 authorisation
  • Fallback AI service
  • Round Robin AI services
  • Lowest Latency AI service
  • Circuit breakers, and backoff-retry over downstream AI services
  • Request based and Token based rate limiting
  • Prompt and usage logging
    • Works for streaming endpoints as-well as non streaming

Configuration

See docs on Github for more details.

Minimal

This sample produces a AI-Central proxy that

  • Listens on a hostname of your choosing
  • Proxies directly through to a back-end Open AI server
  • Can be accessed using standard SDKs
{
  "AICentral": {
    "Endpoints": [
      {
        "Type": "AzureOpenAIEndpoint",
        "Name": "openai-1",
        "Properties": {
          "LanguageEndpoint": "https://<my-ai>.openai.azure.com"
        }
      }
    ],
    "EndpointSelectors": [
      {
        "Type": "SingleEndpoint",
        "Name": "default",
        "Properties": {
          "Endpoint": "openai-1"
        }
      }
    ],
    "Pipelines": [
      {
        "Name": "OpenAIPipeline",
        "Host": "mypipeline.mydomain.com",
        "EndpointSelector": "default"
      }
    ]
  }
}

Full example

This pipeline will:

  • Present an Azure Open AI, and an Open AI downstream as a single upstream endpoint
    • maps incoming Azure Open AI deployments to Open AI models
  • Present it as an Azure Open AI style endpoint
  • Protect the front-end by requiring an AAD token issued for your own AAD application
  • Put a local Asp.Net core rate-limiting policy over the endpoint
  • Add logging to Azure monitor
    • Logs quota, client caller information, and in this case the Prompt but not the response.
{
  "AICentral": {
    "Endpoints": [
      {
        "Type": "AzureOpenAIEndpoint",
        "Name": "openai-priority",
        "Properties": {
          "LanguageEndpoint": "https://<my-ai>.openai.azure.com",
          "AuthenticationType": "Entra|EntraPassThrough|ApiKey",
          "MaxConcurrency": 10
        }
      },
      {
        "Type": "OpenAIEndpoint",
        "Name": "openai-fallback",
        "Properties": {
          "LanguageEndpoint": "https://api.openai.com",
          "ModelMappings": {
            "Gpt35Turbo0613": "gpt-3.5-turbo",
            "Ada002Embedding": "text-embedding-ada-002"
          },
          "ApiKey": "<my-api-key>",
          "Organization": "<optional-organisation>"
        }
      }
    ],
    "AuthProviders": [
      {
        "Type": "Entra",
        "Name": "simple-aad",
        "Properties": {
          "ClientId": "<my-client-id>",
          "TenantId": "<my-tenant-id>",
          "Instance": "https://login.microsoftonline.com/",
          "Audience": "<custom-audience>"
        }
      }
    ],
    "EndpointSelectors": [
      {
        "Type": "Prioritised",
        "Name": "my-endpoint-selector",
        "Properties": {
          "PriorityEndpoints": ["openai-1"],
          "FallbackEndpoints": ["openai-fallback"]
        }
      }
    ],
    "GenericSteps": [
      {
        "Type": "AspNetCoreFixedWindowRateLimiting",
        "Name": "token-rate-limiter",
        "Properties": {
          "LimitType": "PerConsumer|PerAICentralEndpoint",
          ~~"MetricType": "Tokens",
          "Options": {~~
            "Window": 60,
            "PermitLimit": 1000
          }
        }
      },
      {
        "Type": "AspNetCoreFixedWindowRateLimiting",
        "Name": "window-rate-limiter",
        "Properties": {
          "LimitType": "PerConsumer|PerAICentralEndpoint",
          "MetricType": "Requests",
          "Options": {
            "Window": 10,
            "PermitLimit": 100
          }
        }
      },
      {
        "Type": "AzureMonitorLogger",
        "Name": "azure-monitor-logger",
        "Properties": {
          "WorkspaceId": "<workspace-id>",
          "Key": "<key>",
          "LogPrompt": true,
          "LogResponse": false
        }
      },
      {
        "Type": "BulkHead",
        "Name": "bulk-head",
        "Properties": {
          "MaxConcurrency": 20
        }
      }
    ],
    "Pipelines": [
      {
        "Name": "MyPipeline",
        "Host": "prioritypipeline.mydomain.com",
        "EndpointSelector": "my-endpoint-selector",
        "AuthProvider": "simple-aad",
        "Steps": [
          "window-rate-limiter",
          "bulk-head",
          "azure-monitor-logger"
        ]
      }
    ]
  }
}

Product Compatible and additional computed target framework versions.
.NET net7.0 is compatible.  net7.0-android was computed.  net7.0-ios was computed.  net7.0-maccatalyst was computed.  net7.0-macos was computed.  net7.0-tvos was computed.  net7.0-windows was computed.  net8.0 was computed.  net8.0-android was computed.  net8.0-browser was computed.  net8.0-ios was computed.  net8.0-maccatalyst was computed.  net8.0-macos was computed.  net8.0-tvos was computed.  net8.0-windows was computed.  net9.0 was computed.  net9.0-android was computed.  net9.0-browser was computed.  net9.0-ios was computed.  net9.0-maccatalyst was computed.  net9.0-macos was computed.  net9.0-tvos was computed.  net9.0-windows was computed.  net10.0 was computed.  net10.0-android was computed.  net10.0-browser was computed.  net10.0-ios was computed.  net10.0-maccatalyst was computed.  net10.0-macos was computed.  net10.0-tvos was computed.  net10.0-windows was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages

This package is not used by any NuGet packages.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last updated
0.21.4 879 2/21/2025
0.21.4-pullrequest0155-0003 66 2/21/2025
0.21.3 78 2/21/2025
0.21.3-pullrequest0155-0002 65 1/30/2025
0.21.1 602 1/14/2025
0.21.1-pullrequest0153-0003 49 1/14/2025
0.21.0 371 12/9/2024
0.21.0-pullrequest0152-0005 61 12/9/2024
0.21.0-pullrequest0152-0004 61 12/9/2024
0.21.0-pullrequest0152-0003 59 12/9/2024
0.21.0-pullrequest0152-0002 59 12/9/2024
0.20.7 231 11/26/2024
0.20.7-pullrequest0151-0002 63 11/26/2024
0.20.6 369 11/1/2024
0.20.6-pullrequest0150-0003 58 11/1/2024
0.20.5 261 9/30/2024
0.20.5-pullrequest0148-0004 71 9/30/2024
0.20.5-pullrequest0148-0003 67 9/30/2024
0.20.5-pullrequest0148-0002 87 9/30/2024
0.20.3 118 9/5/2024
0.20.3-pullrequest0146-0005 60 9/5/2024
0.20.3-pullrequest0146-0004 63 9/5/2024
0.20.2 84 9/4/2024
0.20.2-pullrequest0145-0004 53 9/4/2024
0.20.0 108 8/17/2024
0.20.0-pullrequest0143-0013 77 8/17/2024
0.20.0-pullrequest0143-0012 76 8/17/2024
0.20.0-pullrequest0143-0011 77 8/17/2024
0.20.0-pullrequest0143-0010 75 8/17/2024
0.20.0-pullrequest0143-0009 74 8/17/2024
0.19.6-pullrequest0143-0008 83 8/17/2024
0.19.5 77 8/7/2024
0.19.5-pullrequest0141-0002 56 8/7/2024
0.19.4 80 8/1/2024
0.19.4-pullrequest0140-0003 58 8/1/2024
0.19.4-pullrequest0139-0001 57 7/29/2024
0.19.3 75 7/29/2024
0.19.2 80 7/29/2024
0.19.2-pullrequest0138-0004 58 7/29/2024
0.19.1 74 7/29/2024
0.19.1-pullrequest0137-0003 59 7/29/2024
0.19.0 69 7/29/2024
0.19.0-pullrequest0136-0014 56 7/29/2024
0.19.0-pullrequest0136-0013 56 7/29/2024
0.19.0-pullrequest0136-0012 146 7/29/2024
0.19.0-pullrequest0136-0011 58 7/29/2024
0.19.0-pullrequest0136-0010 60 7/29/2024
0.19.0-pullrequest0136-0009 71 7/27/2024
0.19.0-pullrequest0136-0008 63 7/27/2024
0.19.0-pullrequest0136-0007 64 7/27/2024
0.18.6 91 7/15/2024
0.18.6-pullrequest0135-0002 59 7/15/2024
0.18.5 77 7/15/2024
0.18.5-pullrequest0134-0003 56 7/15/2024
0.18.4 79 7/15/2024
0.18.4-pullrequest0133-0009 61 7/15/2024
0.18.4-pullrequest0133-0008 57 7/15/2024
0.18.4-pullrequest0133-0007 61 7/15/2024
0.18.3 85 7/12/2024
0.18.3-pullrequest0132-0005 60 7/12/2024
0.18.2 78 7/12/2024
0.18.2-pullrequest0131-0003 68 7/11/2024
0.18.2-pullrequest0130-0003 61 7/11/2024
0.18.1 78 7/10/2024
0.18.1-pullrequest0129-0007 58 7/10/2024
0.18.0 123 7/10/2024
0.18.0-pullrequest0125-0014 60 7/10/2024
0.18.0-pullrequest0125-0013 67 7/10/2024
0.18.0-pullrequest0125-0012 55 7/10/2024
0.18.0-pullrequest0125-0011 58 7/10/2024
0.18.0-pullrequest0125-0010 61 7/10/2024
0.18.0-pullrequest0125-0009 71 7/10/2024
0.17.2-pullrequest0125-0008 62 7/10/2024
0.17.2-pullrequest0125-0007 66 7/10/2024
0.17.2-pullrequest0125-0006 62 7/10/2024
0.17.2-pullrequest0125-0005 64 7/10/2024
0.17.2-pullrequest0125-0004 66 7/10/2024
0.17.2-pullrequest0125-0003 63 7/9/2024
0.17.1 79 7/9/2024
0.17.1-pullrequest0124-0005 64 7/9/2024
0.17.0 84 7/8/2024
0.17.0-pullrequest0123-0024 52 7/8/2024
0.17.0-pullrequest0123-0023 53 7/8/2024
0.17.0-pullrequest0123-0021 68 7/8/2024
0.17.0-pullrequest0123-0019 59 7/8/2024
0.17.0-pullrequest0123-0018 51 7/8/2024
0.17.0-pullrequest0123-0017 63 7/8/2024
0.17.0-pullrequest0123-0015 59 7/8/2024
0.17.0-pullrequest0123-0014 55 7/8/2024
0.17.0-pullrequest0123-0013 60 7/8/2024
0.17.0-pullrequest0123-0012 54 7/8/2024
0.17.0-pullrequest0123-0011 62 7/8/2024
0.17.0-pullrequest0123-0009 64 7/6/2024
0.17.0-pullrequest0123-0008 58 7/6/2024
0.17.0-pullrequest0123-0007 98 7/6/2024
0.16.23 82 6/12/2024
0.16.23-pullrequest0118-0005 67 6/12/2024
0.16.23-pullrequest0118-0004 59 6/12/2024
0.16.23-pullrequest0118-0003 60 6/12/2024
0.16.23-pullrequest0118-0002 59 6/12/2024
0.16.21 139 5/29/2024
0.16.21-pullrequest0114-0002 65 5/29/2024
0.16.20 85 5/28/2024
0.16.20-pullrequest0112-0002 59 5/28/2024
0.16.19 90 5/13/2024
0.16.19-pullrequest0110-0004 74 5/13/2024
0.16.19-pullrequest0110-0003 63 5/13/2024
0.16.17 106 4/22/2024
0.16.17-pullrequest0108-0005 74 4/19/2024
0.16.17-pullrequest0108-0004 67 4/19/2024
0.16.17-pullrequest0108-0003 66 4/19/2024
0.16.16 98 4/16/2024
0.16.16-pullrequest0107-0003 80 4/16/2024
0.16.15 95 4/16/2024
0.16.15-pullrequest0106-0004 68 4/16/2024
0.16.15-pullrequest0106-0003 65 4/16/2024
0.16.15-pullrequest0106-0002 76 4/16/2024
0.16.14 156 4/16/2024
0.16.13 107 3/23/2024
0.16.13-pullrequest0103-0003 83 3/23/2024
0.16.12 91 3/23/2024
0.16.12-pullrequest0102-0006 73 3/22/2024
0.16.11 92 3/21/2024
0.16.11-pullrequest0101-0004 65 3/21/2024
0.16.11-pullrequest0101-0003 72 3/21/2024
0.16.11-pullrequest0101-0002 64 3/21/2024
0.16.9 98 3/16/2024
0.16.9-pullrequest0098-0008 68 3/16/2024
0.16.9-pullrequest0098-0007 71 3/16/2024
0.16.8 118 3/15/2024
0.16.8-pullrequest0097-0002 68 3/15/2024
0.16.7 94 3/10/2024
0.16.7-pullrequest0095-0002 78 3/10/2024
0.16.6 175 3/9/2024
0.16.6-pullrequest0094-0003 69 3/9/2024
0.16.6-pullrequest0094-0002 67 3/9/2024
0.16.2 103 3/5/2024
0.16.2-pullrequest0089-0009 75 3/5/2024
0.16.1 101 3/3/2024
0.16.1-pullrequest0088-0003 71 3/2/2024
0.16.1-pullrequest0088-0002 61 3/2/2024
0.16.0 145 2/28/2024
0.16.0-pullrequest0087-0010 63 2/28/2024
0.16.0-pullrequest0087-0009 67 2/28/2024
0.16.0-pullrequest0087-0007 77 2/28/2024
0.16.0-pullrequest0087-0006 67 2/28/2024
0.16.0-pullrequest0087-0005 63 2/28/2024
0.16.0-pullrequest0087-0004 74 2/28/2024
0.16.0-pullrequest0087-0003 66 2/28/2024
0.15.2 90 2/26/2024
0.15.2-pullrequest0086-0007 68 2/26/2024
0.15.2-pullrequest0086-0006 58 2/26/2024
0.15.1 98 2/23/2024
0.15.1-pullrequest0085-0006 68 2/23/2024
0.15.0 98 2/22/2024
0.15.0-pullrequest0084-0004 71 2/22/2024
0.15.0-pullrequest0084-0003 99 2/22/2024
0.14.1-pullrequest0084-0002 82 2/22/2024
0.14.0 109 2/21/2024
0.14.0-pullrequest0083-0011 92 2/8/2024
0.14.0-pullrequest0083-0010 73 2/8/2024
0.14.0-pullrequest0083-0009 74 2/8/2024
0.13.7 117 2/8/2024
0.13.7-pullrequest0082-0002 81 2/8/2024
0.13.6-pullrequest0081-0006 73 2/8/2024
0.13.5 123 2/7/2024
0.13.5-pullrequest0080-0002 74 2/7/2024
0.13.4 90 2/7/2024
0.13.4-pullrequest0079-0003 67 2/7/2024
0.13.1 100 2/1/2024
0.13.1-pullrequest0076-0003 75 2/1/2024
0.13.1-pullrequest0076-0002 75 2/1/2024
0.13.0 116 2/1/2024
0.13.0-pullrequest0075-0005 72 2/1/2024
0.13.0-pullrequest0075-0004 66 2/1/2024
0.12.2-pullrequest0075-0003 67 2/1/2024
0.12.1 92 2/1/2024
0.12.1-pullrequest0074-0002 70 2/1/2024
0.12.0 90 2/1/2024
0.12.0-pullrequest0073-0002 67 2/1/2024
0.11.0 91 1/31/2024
0.11.0-pullrequest0072-0011 71 1/31/2024
0.11.0-pullrequest0072-0010 69 1/31/2024
0.11.0-pullrequest0072-0008 68 1/31/2024
0.10.0 94 1/31/2024
0.10.0-pullrequest0070-0005 75 1/29/2024
0.10.0-pullrequest0070-0004 70 1/29/2024
0.9.1-pullrequest0069-0001 70 1/29/2024
0.9.0 88 1/29/2024
0.9.0-pullrequest0069-0002 67 1/29/2024
0.8.7-pullrequest0068-0001 70 1/29/2024
0.8.6 92 1/29/2024
0.8.4 110 1/19/2024
0.8.4-pullrequest0066-0009 71 1/19/2024
0.8.4-pullrequest0066-0008 62 1/19/2024
0.8.4-pullrequest0066-0007 69 1/19/2024
0.8.4-pullrequest0066-0005 65 1/19/2024
0.8.4-pullrequest0066-0004 73 1/19/2024
0.8.4-pullrequest0066-0003 67 1/19/2024
0.8.3 87 1/19/2024
0.8.3-pullrequest0065-0003 69 1/19/2024
0.8.3-pullrequest0065-0002 68 1/19/2024
0.8.1 90 1/19/2024
0.8.1-pullrequest0063-0013 67 1/19/2024
0.8.1-pullrequest0063-0011 71 1/19/2024
0.7.16-pullrequest0063-0010 76 1/19/2024
0.7.15 97 1/18/2024
0.7.15-pullrequest0062-0002 72 1/18/2024
0.7.12 104 1/16/2024
0.7.12-pullrequest0059-0006 75 1/16/2024
0.7.10 97 1/15/2024
0.7.10-pullrequest0059-0007 71 1/16/2024
0.7.10-pullrequest0058-0004 65 1/15/2024
0.7.10-pullrequest0057-0002 71 1/15/2024
0.7.9 97 1/15/2024
0.7.9-pullrequest0057-0003 66 1/15/2024
0.7.7 93 1/15/2024
0.7.7-pullrequest0056-0003 81 1/15/2024
0.7.4 96 1/15/2024
0.7.4-pullrequest0055-0012 74 1/15/2024
0.7.4-pullrequest0055-0011 74 1/15/2024
0.7.4-pullrequest0055-0010 79 1/15/2024
0.7.4-pullrequest0055-0009 73 1/15/2024
0.7.4-pullrequest0055-0008 72 1/15/2024
0.7.4-pullrequest0055-0007 76 1/15/2024
0.7.4-pullrequest0054-0005 75 1/15/2024
0.7.4-pullrequest0053-0004 87 1/15/2024
0.7.4-pullrequest0052-0002 68 1/15/2024
0.7.2 92 1/15/2024
0.7.2-pullrequest0051-0013 76 1/15/2024
0.7.2-pullrequest0050-0010 66 1/15/2024
0.7.1 103 1/13/2024
0.7.0 100 1/13/2024
0.7.0-pullrequest0046-0079 86 1/13/2024
0.6.12 120 1/3/2024
0.6.12-pullrequest0045-0018 72 1/13/2024
0.6.12-pullrequest0042-0010 75 1/12/2024
0.6.12-pullrequest0041-0007 74 1/12/2024
0.6.12-pullrequest0039-0002 80 1/3/2024
0.6.10 115 12/22/2023
0.6.10-pullrequest0038-0007 98 1/2/2024
0.6.10-pullrequest0038-0006 81 1/2/2024
0.6.10-pullrequest0038-0005 81 1/2/2024
0.6.8 110 12/21/2023
0.6.6 105 12/18/2023
0.6.6-pullrequest0035-0007 85 12/21/2023
0.6.6-pullrequest0035-0006 86 12/21/2023
0.6.6-pullrequest0035-0005 80 12/21/2023
0.6.3 104 12/18/2023
0.6.3-pullrequest0034-0018 84 12/18/2023
0.6.1 98 12/15/2023
0.6.1-pullrequest0029-0008 88 12/15/2023
0.6.1-pullrequest0028-0006 90 12/15/2023
0.6.1-pullrequest0027-0004 68 12/15/2023
0.5.3 107 12/11/2023
0.5.3-pullrequest0025-0049 88 12/14/2023
0.5.3-pullrequest0025-0047 85 12/14/2023
0.5.3-pullrequest0025-0043 80 12/13/2023
0.5.3-pullrequest0022-0038 81 12/13/2023
0.5.1 94 12/7/2023
0.5.0 100 12/6/2023
0.4.3 106 12/4/2023
0.4.2 97 12/4/2023
0.4.1 107 12/4/2023
0.3.10 115 11/22/2023
0.3.10-pullrequest0022-0008 82 11/27/2023
0.3.10-pullrequest0021-0005 85 11/22/2023
0.3.9 93 11/20/2023
0.3.8 104 11/9/2023
0.3.8-pullrequest0020-0005 80 11/9/2023
0.3.6 95 11/9/2023
0.3.5 102 11/9/2023
0.3.4 97 11/8/2023
0.3.3 99 11/8/2023
0.3.1 101 11/8/2023
0.3.1-pullrequest0018-0004 88 11/8/2023
0.2.1 135 11/2/2023
0.2.1-pullrequest0017-0050 97 11/8/2023
0.2.1-pullrequest0016-0048 99 11/8/2023
0.2.1-pullrequest0015-0045 88 11/8/2023
0.2.1-pullrequest0014-0035 98 11/7/2023
0.2.1-pullrequest0013-0033 89 11/7/2023
0.2.1-pullrequest0012-0031 94 11/7/2023
0.2.1-pullrequest0011-0024 92 11/6/2023
0.2.1-pullrequest0010-0015 105 11/2/2023
0.2.1-pullrequest0009-0011 95 11/2/2023
0.2.1-pullrequest0008-0009 96 11/2/2023
0.2.1-pullrequest0007-0007 103 11/2/2023
0.2.1-pullrequest0006-0005 98 11/2/2023
0.2.1-pullrequest0005-0001 89 11/2/2023
0.1.0 123 10/27/2023
0.1.0-pullrequest0004-0024 98 11/1/2023
0.1.0-pullrequest0003-0022 99 11/1/2023
0.1.0-pullrequest0002-0019 97 11/1/2023
0.1.0-pullrequest0001-0017 99 11/1/2023
0.1.0-pullrequest0001-0010 100 10/30/2023