LLMSharp.Anthropic 1.0.2

anthropic-sdk-dotnet

C# Client SDK (Unofficial) for Anthropic Large Language Models

• Inspired by the official Anthropic Python SDK.
• Goal is to provide the efficient and flexible SDK for dotnet developers building LLM Apps using Anthropic
• Includes methods for fetching UsageInfo using AnthropicTokenizer.
• SDK uses http/2 for calling Anthropic API endpoints when the client code targets .NET Core 3.0/3.1 or .NET >= 5, for other versions the SDK falls back to using http/1.1

Install 💽

dotnet add package LLMSharp.Anthropic

Usage

Quickstart 🚀

• Create ClientOptions object with Anthropic API key

using LLMSharp.Anthropic;

// if you want to explicitly pass the api key
ClientOptions options = new ClientOptions { ApiKey = <your anthropic api key> };

OR

using LLMSharp.Anthropic;

// set API key as an environment variable with key 'ANTHROPIC_API_KEY'
ClientOptions options = new ClientOptions();

• Create an instance of AnthropicClient and pass in the client Options.

using LLMSharp.Anthropic;

AnthropicClient client = new AnthropicClient(options);

• Get non streaming chat completions for a prompt

using LLMSharp.Anthropic;
using LLMSharp.Anthropic.Models;

string prompt = $"{Constants.HUMAN_PROMPT}<your custom prompt will go here>{Constants.AI_PROMPT}";
AnthropicCreateNonStreamingCompletionParams reqParams = new AnthropicCreateNonStreamingCompletionParams { Prompt = prompt };
AnthropicCompletion completion = await client.GetCompletionsAsync(reqParams);
Console.WriteLine(completion.Completion);

• Get streaming chat completions for a prompt

using LLMSharp.Anthropic;
using LLMSharp.Anthropic.Models;

string prompt = $"{Constants.HUMAN_PROMPT}<your custom prompt will go here>{Constants.AI_PROMPT}";
AnthropicCreateStreamingCompletionParams reqParams = new AnthropicCreateStreamingCompletionParams { Prompt = prompt };
IAsyncEnuemrable<AnthropicCompletion> completions = await client.GetStreamingCompletionsAsync(reqParams);

await using(AnthropicCompletion completion in completions)
{
    Console.WriteLine(completion.Completion);
}

TLDR 🎯

AnthropicClient Methods :

• GetCompletionsAsync : Get non streaming completions, returns an AnthropicCompletion object.

• GetRawCompletionsAsync : Get non stream completions as raw httpresponse message.

• GetCompletionsWithUsageInfoAsync : Get non streaming completions with token usage info (prompt tokens , completion tokens), returns an AnthropicCompletion object with usage info.

• GetStreamingCompletionsAsync : Get streaming completions, returns an IAsyncEnumerable stream of AnthropicCompletion objects.

• GetStreamingCompletionsAsStreamAsync : Get streaming completions as a raw stream, returns a SSE stream.

• GetRawStreamingCompletionsAsync : Get streaming completions raw httpresponse message.

• GetStreamingCompletionsWithUsageInfoAsync : Get streaming completions with token usage info (prompt tokens , completion tokens), returns an AnthropicCompletion object with usage info.

Input models :

• AnthropicCreateNonStreamingCompletionParams : is used for passing input parameters for non streaming completion methods.
• AnthropicCreateNonStreamingCompletionParams : is used for passing input parameters for streaming completion methods.
• ClientOptions : Provides various options for configuring AnthropicClient instance
• AnthropicRequestOptions : Provides options for overriding AnthropicClient configuration with request specific configuration

Advanced Usage 📋

• I want to control additional attributes like 'Temperature', 'TopP' and 'TopK' ⇒ customize using AnthropicCreateStreamingCompletionParams and AnthropicCreateNonStreamingCompletionParams for Streaming and non streaming completions respectively

var userQuestion = "Why is the sky blue?";
var prompt = $"{Constants.HUMAN_PROMPT}{userQuestion}{Constants.AI_PROMPT}";

AnthropicCreateStreamingCompletionParams reqParams = 
    new AnthropicCreateStreamingCompletionParams
    {
        /*
        * The maximum number of tokens to generate before stopping.
        * Default : 2048
        */
        MaxTokens = 2048,
        /*
        * The model that will complete your prompt.
        * Possible Options: claude-2 , claude-instant-1, your custom model name
        * Default: "claude-2"
        */
        Model = "claude-2",
        /*
        * The prompt that you want Claude to complete.
        * Default : "\n\nHuman:Hello,Claude. \n\nAssistant:"
        * See [comments on prompts](https://docs.anthropic.com/claude/docs/introduction-to-prompt-design)
        */
        Prompt = prompt,
        /*
        * Sequences that will cause the model to stop generating completion text.
        * Anthropic models stop on `"\n\nHuman:"`, and may include additional built-in stop 
        sequences in the future. By providing the stop_sequences parameter, you may include 
        additional strings that will cause the model to stop generating.
        */
        StopSequences = new string[]{""},
        /*
        * Amount of randomness injected into the response.
        * Defaults to 1.
        * Ranges from 0 to 1. Use temp closer to 0 for analytical / multiple choice,
        * and closer to 1 for creative and generative tasks.
        */
        Temperature = 1,
        /*
        * Use nucleus sampling.In nucleus sampling, we compute the cumulative distribution 
        * over all the options for each subsequent token in decreasing probability order 
        * and cut it off once it reaches a particular probability specified by `TopP`. You 
        * should either alter Temperature or TopP , but not both
        */       
        TopP = null,
        /*
        * Only sample from the top K options for each subsequent token.
        * Used to remove "long tail" low probability responses.
        * Default 'null'
        */
        TopK = null,

    };

var completions = await client.GetStreamingCompletionsAsync(reqParams);

// Use 'AnthropicCreateNonStreamingCompletionParams' and GetCompletionsAsync if looking for NonStreaming completions

• I want the raw http response that the anthropic api returned.

  • Non streaming completions get httpresponsemessage ⇒ use GetRawCompletionsAsync

  using LLMSharp.Anthropic;
  HttpResponseMessage message = await client.GetRawCompletionsAsync(options);
  

  • Streaming completions get httpresponsemessage ⇒ use GetRawStreamingCompletionsAsync

  using LLMSharp.Anthropic;
  HttpResponseMessage message = await client.GetRawStreamingCompletionsAsync(options);
  

  • Streaming completions get raw stream ⇒ use GetStreamingCompletionsAsStreamAsync

  using LLMSharp.Anthropic;
  Stream stream = await client.GetStreamingCompletionsAsStreamAsync(options);
  

• I want more control when creating AnthropicClient ⇒ customize ClientOptions

using LLMSharp.Anthropic;

ClientOptions options = new ClientOptions
{
    ApiKey = "<apikey>",
    BaseUrl = "a different baseurl other than default",
    DefaultHeaders = new Dictionary<string, IEnumerable<string>>
    {
        "header1", new string[]{"val1", "val2"}
    },
    MaxRetries = 3, // max retries
    Timeout = 500 // 500 ms
};

AnthropicClient client = new AnthropicClient(options);

• I want to override 'MaxRetries' and 'Timeout' on a per request basis , instead of those configured using ClientOptions for the AnthropicClient ⇒ Use AnthropicRequestOptions

using LLMSharp.Anthropic;
using LLMSharp.Anthropic.Models;

var userQuestion = "Why is the sky blue?";
var prompt = $"{Constants.HUMAN_PROMPT}{userQuestion}{Constants.AI_PROMPT}";

AnthropicCreateStreamingCompletionParams reqParams = new AnthropicCreateStreamingCompletionParams{Prompt = prompt};

AnthropicRequestOptions reqOptions = new AnthropicRequestOptions
{
    MaxRetries = 5, // overrides max number of retries for the request to 5
    Timeout = 500 // overrides timeout per retry for this request to 500 ms
};

var completions = await client.GetStreamingCompletionsAsync(reqParams, reqOptions);

• I want to override 'Anthropic Api Key' or 'Auth Token' on a per request basis ⇒ Use 'RequestHeaders' properties of AnthropicRequestOptions to override.

using LLMSharp.Anthropic;
using LLMSharp.Anthropic.Models;

var userQuestion = "Why is the sky blue?";
var prompt = $"{Constants.HUMAN_PROMPT}{userQuestion}{Constants.AI_PROMPT}";

AnthropicCreateStreamingCompletionParams reqParams = new AnthropicCreateStreamingCompletionParams{Prompt = prompt};

// override 'x-api-key' request header for this request
AnthropicRequestOptions reqOptions = new AnthropicRequestOptions
{
    RequestHeaders = new Dictionary<string, IEnumerable<string>>
    {
        "X-API-KEY", new string[]{"your new api key"}
    }
};

var completions = await client.GetStreamingCompletionsAsync(reqParams, reqOptions);

Use 'Authorization' header if needs to override AuthToken

using LLMSharp.Anthropic;
using LLMSharp.Anthropic.Models;

var userQuestion = "Why is the sky blue?";
var prompt = $"{Constants.HUMAN_PROMPT}{userQuestion}{Constants.AI_PROMPT}";

AnthropicCreateStreamingCompletionParams reqParams = new AnthropicCreateStreamingCompletionParams{Prompt = prompt};

// override 'x-api-key' request header for this request
AnthropicRequestOptions reqOptions = new AnthropicRequestOptions
{
    RequestHeaders = new Dictionary<string, IEnumerable<string>>
    {
        "Authorization", new string[]{"Bearer <your new auth token>"}
    }
};

var completions = await client.GetStreamingCompletionsAsync(reqParams, reqOptions);

Contribute 🤝

Got a pull request? Open it, and I'll review it as soon as possible.