OpenAI-DotNet
6.0.0
See the version list below for details.
dotnet add package OpenAI-DotNet --version 6.0.0
NuGet\Install-Package OpenAI-DotNet -Version 6.0.0
<PackageReference Include="OpenAI-DotNet" Version="6.0.0" />
paket add OpenAI-DotNet --version 6.0.0
#r "nuget: OpenAI-DotNet, 6.0.0"
// Install OpenAI-DotNet as a Cake Addin #addin nuget:?package=OpenAI-DotNet&version=6.0.0 // Install OpenAI-DotNet as a Cake Tool #tool nuget:?package=OpenAI-DotNet&version=6.0.0
C#/.NET SDK for accessing the OpenAI GPT-3 API
A simple C# .NET wrapper library to use with OpenAI's GPT-3 API. More context on Roger Pincombe's blog and forked from OpenAI-API-dotnet.
This repository is available to transfer to the OpenAI organization if they so choose to accept it.
Requirements
This library targets .NET 6.0 and above.
It should work across console apps, winforms, wpf, asp.net, etc.
It should also work across Windows, Linux, and Mac.
Getting started
Install from NuGet
Install package OpenAI
from Nuget. Here's how via command line:
Install-Package OpenAI-DotNet
Looking to use OpenAI in the Unity Game Engine? Check out our unity package on OpenUPM:
Documentation
Table of Contents
- Authentication
- Azure OpenAI
- Models
- Completions
- Chat
- Edits
- Embeddings
- Audio
- Images
- Files
- Fine Tuning
- Moderations
Authentication
There are 3 ways to provide your API keys, in order of precedence:
- Pass keys directly with constructor
- Load key from configuration file
- Use System Environment Variables
You use the OpenAIAuthentication
when you initialize the API as shown:
Pass keys directly with constructor
var api = new OpenAIClient("sk-apiKey");
Or create a OpenAIAuthentication
object manually
var api = new OpenAIClient(new OpenAIAuthentication("sk-apiKey", "org-yourOrganizationId"));
Load key from configuration file
Attempts to load api keys from a configuration file, by default .openai
in the current directory, optionally traversing up the directory tree or in the user's home directory.
To create a configuration file, create a new text file named .openai
and containing the line:
Organization entry is optional.
Json format
{
"apiKey": "sk-aaaabbbbbccccddddd",
"organization": "org-yourOrganizationId"
}
Deprecated format
OPENAI_KEY=sk-aaaabbbbbccccddddd
ORGANIZATION=org-yourOrganizationId
You can also load the file directly with known path by calling a static method in Authentication:
var api = new OpenAIClient(OpenAIAuthentication.LoadFromDirectory("your/path/to/.openai"));;
Use System Environment Variables
Use your system's environment variables specify an api key and organization to use.
- Use
OPENAI_API_KEY
for your api key. - Use
OPENAI_ORGANIZATION_ID
to specify an organization.
var api = new OpenAIClient(OpenAIAuthentication.LoadFromEnv());
Azure OpenAI
You can also choose to use Microsoft's Azure OpenAI deployments as well.
To setup the client to use your deployment, you'll need to pass in OpenAIClientSettings
into the client constructor.
var auth = new OpenAIAuthentication("sk-apiKey");
var settings = new OpenAIClientSettings("your-resource", "your-deployment-id");
var api = new OpenAIClient(auth, settings);
Models
List and describe the various models available in the API. You can refer to the Models documentation to understand what models are available and the differences between them.
The Models API is accessed via OpenAIClient.ModelsEndpoint
List models
Lists the currently available models, and provides basic information about each one such as the owner and availability.
var api = new OpenAIClient();
var models = await api.ModelsEndpoint.GetModelsAsync();
foreach (var model in models)
{
Console.WriteLine(model.ToString());
}
Retrieve model
Retrieves a model instance, providing basic information about the model such as the owner and permissioning.
var api = new OpenAIClient();
var model = await api.ModelsEndpoint.GetModelDetailsAsync("text-davinci-003");
Console.WriteLine(model.ToString());
Delete Fine Tuned Model
Delete a fine-tuned model. You must have the Owner role in your organization.
var api = new OpenAIClient();
var result = await api.ModelsEndpoint.DeleteFineTuneModelAsync("your-fine-tuned-model");
Assert.IsTrue(result);
Completions
Given a prompt, the model will return one or more predicted completions, and can also return the probabilities of alternative tokens at each position.
var api = new OpenAIClient();
var result = await api.CompletionsEndpoint.CreateCompletionAsync("One Two Three One Two", temperature: 0.1, model: Model.Davinci);
Console.WriteLine(result);
To get the
CompletionResult
(which is mostly metadata), use its implicit string operator to get the text if all you want is the completion choice.
Completion Streaming
Streaming allows you to get results are they are generated, which can help your application feel more responsive, especially on slow models like Davinci.
var api = new OpenAIClient();
await api.CompletionsEndpoint.StreamCompletionAsync(result =>
{
foreach (var choice in result.Completions)
{
Console.WriteLine(choice);
}
}, "My name is Roger and I am a principal software engineer at Salesforce. This is my resume:", maxTokens: 200, temperature: 0.5, presencePenalty: 0.1, frequencyPenalty: 0.1, model: Model.Davinci);
Or if using IAsyncEnumerable{T}
(C# 8.0+)
var api = new OpenAIClient();
await foreach (var token in api.CompletionsEndpoint.StreamCompletionEnumerableAsync("My name is Roger and I am a principal software engineer at Salesforce. This is my resume:", maxTokens: 200, temperature: 0.5, presencePenalty: 0.1, frequencyPenalty: 0.1, model: Model.Davinci))
{
Console.WriteLine(token);
}
Chat
Given a chat conversation, the model will return a chat completion response.
Chat Completions
Creates a completion for the chat message
var api = new OpenAIClient();
var chatPrompts = new List<ChatPrompt>
{
new ChatPrompt("system", "You are a helpful assistant."),
new ChatPrompt("user", "Who won the world series in 2020?"),
new ChatPrompt("assistant", "The Los Angeles Dodgers won the World Series in 2020."),
new ChatPrompt("user", "Where was it played?"),
};
var chatRequest = new ChatRequest(chatPrompts);
var result = await api.ChatEndpoint.GetCompletionAsync(chatRequest);
Console.WriteLine(result.FirstChoice);
Chat Streaming
var api = new OpenAIClient();
var chatPrompts = new List<ChatPrompt>
{
new ChatPrompt("system", "You are a helpful assistant."),
new ChatPrompt("user", "Who won the world series in 2020?"),
new ChatPrompt("assistant", "The Los Angeles Dodgers won the World Series in 2020."),
new ChatPrompt("user", "Where was it played?"),
};
var chatRequest = new ChatRequest(chatPrompts, Model.GPT3_5_Turbo);
await api.ChatEndpoint.StreamCompletionAsync(chatRequest, result =>
{
Console.WriteLine(result.FirstChoice);
});
Or if using IAsyncEnumerable{T}
(C# 8.0+)
var api = new OpenAIClient();
var chatPrompts = new List<ChatPrompt>
{
new ChatPrompt("system", "You are a helpful assistant."),
new ChatPrompt("user", "Who won the world series in 2020?"),
new ChatPrompt("assistant", "The Los Angeles Dodgers won the World Series in 2020."),
new ChatPrompt("user", "Where was it played?"),
};
var chatRequest = new ChatRequest(chatPrompts, Model.GPT3_5_Turbo);
await foreach (var result in api.ChatEndpoint.StreamCompletionEnumerableAsync(chatRequest))
{
Console.WriteLine(result.FirstChoice);
}
Edits
Given a prompt and an instruction, the model will return an edited version of the prompt.
The Edits API is accessed via OpenAIClient.EditsEndpoint
Create Edit
Creates a new edit for the provided input, instruction, and parameters using the provided input and instruction.
var api = new OpenAIClient();
var request = new EditRequest("What day of the wek is it?", "Fix the spelling mistakes");
var result = await api.EditsEndpoint.CreateEditAsync(request);
Console.WriteLine(result);
Embeddings
Get a vector representation of a given input that can be easily consumed by machine learning models and algorithms.
Related guide: Embeddings
The Edits API is accessed via OpenAIClient.EmbeddingsEndpoint
Create Embeddings
Creates an embedding vector representing the input text.
var api = new OpenAIClient();
var model = await api.ModelsEndpoint.GetModelDetailsAsync("text-embedding-ada-002");
var result = await api.EmbeddingsEndpoint.CreateEmbeddingAsync("The food was delicious and the waiter...", model);
Console.WriteLine(result);
Audio
Converts audio into text.
Create Transcription
Transcribes audio into the input language.
var api = new OpenAIClient();
var request = new AudioTranscriptionRequest(Path.GetFullPath(audioAssetPath), language: "en");
var result = await api.AudioEndpoint.CreateTranscriptionAsync(request);
Console.WriteLine(result);
Create Translation
Translates audio into into English.
var api = new OpenAIClient();
var request = new AudioTranslationRequest(Path.GetFullPath(audioAssetPath));
var result = await api.AudioEndpoint.CreateTranslationAsync(request);
Console.WriteLine(result);
Images
Given a prompt and/or an input image, the model will generate a new image.
The Images API is accessed via OpenAIClient.ImagesEndpoint
Create Image
Creates an image given a prompt.
var api = new OpenAIClient();
var results = await api.ImagesEndPoint.GenerateImageAsync("A house riding a velociraptor", 1, ImageSize.Small);
foreach (var result in results)
{
Console.WriteLine(result);
// result == file://path/to/image.png
}
Edit Image
Creates an edited or extended image given an original image and a prompt.
var api = new OpenAIClient();
var results = await api.ImagesEndPoint.CreateImageEditAsync(Path.GetFullPath(imageAssetPath), Path.GetFullPath(maskAssetPath), "A sunlit indoor lounge area with a pool containing a flamingo", 1, ImageSize.Small);
foreach (var result in results)
{
Console.WriteLine(result);
// result == file://path/to/image.png
}
Create Image Variation
Creates a variation of a given image.
var api = new OpenAIClient();
var results = await api.ImagesEndPoint.CreateImageVariationAsync(Path.GetFullPath(imageAssetPath), 1, ImageSize.Small);
foreach (var result in results)
{
Console.WriteLine(result);
// result == file://path/to/image.png
}
Files
Files are used to upload documents that can be used with features like Fine-tuning.
The Files API is accessed via OpenAIClient.FilesEndpoint
List Files
Returns a list of files that belong to the user's organization.
var api = new OpenAIClient();
var files = await api.FilesEndpoint.ListFilesAsync();
foreach (var file in files)
{
Console.WriteLine($"{file.Id} -> {file.Object}: {file.FileName} | {file.Size} bytes");
}
Upload File
Upload a file that contains document(s) to be used across various endpoints/features. Currently, the size of all the files uploaded by one organization can be up to 1 GB. Please contact us if you need to increase the storage limit.
var api = new OpenAIClient();
var fileData = await api.FilesEndpoint.UploadFileAsync("path/to/your/file.jsonl", "fine-tune");
Console.WriteLine(fileData.Id);
Delete File
Delete a file.
var api = new OpenAIClient();
var result = await api.FilesEndpoint.DeleteFileAsync(fileData);
Assert.IsTrue(result);
Retrieve File Info
Returns information about a specific file.
var api = new OpenAIClient();
var fileData = await GetFileInfoAsync(fileId);
Console.WriteLine($"{fileData.Id} -> {fileData.Object}: {fileData.FileName} | {fileData.Size} bytes");
Download File Content
Downloads the specified file.
var api = new OpenAIClient();
var downloadedFilePath = await api.FilesEndpoint.DownloadFileAsync(fileId, "path/to/your/save/directory");
Console.WriteLine(downloadedFilePath);
Assert.IsTrue(File.Exists(downloadedFilePath));
Fine Tuning
Manage fine-tuning jobs to tailor a model to your specific training data.
Related guide: Fine-tune models
The Files API is accessed via OpenAIClient.FineTuningEndpoint
Create Fine Tune Job
Creates a job that fine-tunes a specified model from a given dataset.
Response includes details of the enqueued job including job status and the name of the fine-tuned models once complete.
var api = new OpenAIClient();
var request = new CreateFineTuneRequest(fileData);
var fineTuneJob = await api.FineTuningEndpoint.CreateFineTuneJobAsync(request);
Console.WriteLine(fineTuneJob.Id);
List Fine Tune Jobs
List your organization's fine-tuning jobs.
var api = new OpenAIClient();
var fineTuneJobs = await api.FineTuningEndpoint.ListFineTuneJobsAsync();
foreach (var job in fineTuneJobs)
{
Console.WriteLine($"{job.Id} -> {job.Status}");
}
Retrieve Fine Tune Job Info
Gets info about the fine-tune job.
var api = new OpenAIClient();
var result = await api.FineTuningEndpoint.RetrieveFineTuneJobInfoAsync(fineTuneJob);
Console.WriteLine($"{result.Id} -> {result.Status}");
Cancel Fine Tune Job
Immediately cancel a fine-tune job.
var api = new OpenAIClient();
var result = await api.FineTuningEndpoint.CancelFineTuneJobAsync(fineTuneJob);
Assert.IsTrue(result);
List Fine Tune Events
Get fine-grained status updates for a fine-tune job.
var api = new OpenAIClient();
var fineTuneEvents = await api.FineTuningEndpoint.ListFineTuneEventsAsync(fineTuneJob);
Console.WriteLine($"{fineTuneJob.Id} -> status: {fineTuneJob.Status} | event count: {fineTuneEvents.Count}");
Stream Fine Tune Events
var api = new OpenAIClient();
await api.FineTuningEndpoint.StreamFineTuneEventsAsync(fineTuneJob, fineTuneEvent =>
{
Console.WriteLine($" {fineTuneEvent.CreatedAt} [{fineTuneEvent.Level}] {fineTuneEvent.Message}");
});
Or if using IAsyncEnumerable{T}
(C# 8.0+)
var api = new OpenAIClient();
await foreach (var fineTuneEvent in api.FineTuningEndpoint.StreamFineTuneEventsEnumerableAsync(fineTuneJob))
{
Console.WriteLine($" {fineTuneEvent.CreatedAt} [{fineTuneEvent.Level}] {fineTuneEvent.Message}");
}
Moderations
Given a input text, outputs if the model classifies it as violating OpenAI's content policy.
Related guide: Moderations
The Moderations API can be accessed via OpenAIClient.ModerationsEndpoint
Create Moderation
Classifies if text violates OpenAI's Content Policy.
var api = new OpenAIClient();
var response = await api.ModerationsEndpoint.GetModerationAsync("I want to kill them.");
Assert.IsTrue(response);
License
This library is licensed CC-0, in the public domain. You can use it for whatever you want, publicly or privately, without worrying about permission or licensing or whatever. It's just a wrapper around the OpenAI API, so you still need to get access to OpenAI from them directly. I am not affiliated with OpenAI and this library is not endorsed by them, I just have beta access and wanted to make a C# library to access it more easily. Hopefully others find this useful as well. Feel free to open a PR if there's anything you want to contribute.
Product | Versions Compatible and additional computed target framework versions. |
---|---|
.NET | net6.0 is compatible. net6.0-android was computed. net6.0-ios was computed. net6.0-maccatalyst was computed. net6.0-macos was computed. net6.0-tvos was computed. net6.0-windows was computed. net7.0 was computed. net7.0-android was computed. net7.0-ios was computed. net7.0-maccatalyst was computed. net7.0-macos was computed. net7.0-tvos was computed. net7.0-windows was computed. net8.0 was computed. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. |
-
net6.0
- No dependencies.
NuGet packages (11)
Showing the top 5 NuGet packages that depend on OpenAI-DotNet:
Package | Downloads |
---|---|
OpenAI-DotNet-Proxy
A simple Proxy API gateway for OpenAI-DotNet to make authenticated requests from a front end application without exposing your API keys. |
|
WK.OpenAiWrapper
Package Description |
|
NewLeadAI
A client to interact with NewLead AI API |
|
AICore.Core
Package Description |
|
Universal.Tools.Core.Requests.Extensions.OpenAI
Package Description |
GitHub repositories (3)
Showing the top 3 popular GitHub repositories that depend on OpenAI-DotNet:
Repository | Stars |
---|---|
dkgv/pinpoint
Keystroke launcher and personal command central. Alternative to Spotlight and Alfred for Windows. Alternative to Wox, PowerToys.
|
|
SlimeNull/OpenGptChat
An OpenAI Chat completion Client. 一个 OpenAI 聊天 Completion 客户端.
|
|
BoiHanny/vrcosc-magicchatbox
The ultimate companion, whether you're on desktop or in VR, we've got you covered with our handy integrations in a compact and modern UI
|
Version | Downloads | Last updated | |
---|---|---|---|
8.4.1 | 2,756 | 11/15/2024 | |
8.4.0 | 227 | 11/15/2024 | |
8.3.0 | 31,112 | 9/19/2024 | |
8.2.5 | 4,858 | 9/14/2024 | |
8.2.4 | 702 | 9/14/2024 | |
8.2.2 | 14,810 | 8/19/2024 | |
8.2.1 | 338 | 8/19/2024 | |
8.2.0 | 592 | 8/18/2024 | |
8.1.2 | 9,458 | 8/9/2024 | |
8.1.1 | 38,809 | 6/30/2024 | |
8.1.0 | 9,011 | 6/21/2024 | |
8.0.3 | 1,193 | 6/16/2024 | |
8.0.2 | 1,646 | 6/15/2024 | |
8.0.1 | 14,343 | 6/10/2024 | |
8.0.0 | 197 | 6/10/2024 | |
7.7.8 | 47,302 | 5/4/2024 | |
7.7.7 | 16,212 | 4/21/2024 | |
7.7.6 | 15,460 | 3/19/2024 | |
7.7.5 | 14,619 | 3/3/2024 | |
7.7.4 | 1,725 | 2/29/2024 | |
7.7.3 | 889 | 2/27/2024 | |
7.7.2 | 1,299 | 2/27/2024 | |
7.7.1 | 1,862 | 2/25/2024 | |
7.7.0 | 3,418 | 2/22/2024 | |
7.6.5 | 8,765 | 2/6/2024 | |
7.6.4 | 2,297 | 1/29/2024 | |
7.6.3 | 608 | 1/26/2024 | |
7.6.2 | 9,969 | 1/14/2024 | |
7.6.1 | 3,291 | 1/6/2024 | |
7.6.0 | 7,585 | 1/2/2024 | |
7.5.0 | 3,753 | 12/22/2023 | |
7.4.4 | 15,211 | 12/10/2023 | |
7.4.3 | 1,132 | 12/7/2023 | |
7.4.2 | 1,262 | 12/7/2023 | |
7.4.1 | 4,714 | 12/3/2023 | |
7.4.0 | 2,185 | 11/30/2023 | |
7.3.8 | 631 | 11/29/2023 | |
7.3.7 | 859 | 11/28/2023 | |
7.3.6 | 457 | 11/28/2023 | |
7.3.5 | 729 | 11/27/2023 | |
7.3.4 | 4,811 | 11/24/2023 | |
7.3.3 | 1,165 | 11/23/2023 | |
7.3.2 | 876 | 11/22/2023 | |
7.3.1 | 5,201 | 11/21/2023 | |
7.3.0 | 467 | 11/21/2023 | |
7.2.3 | 5,117 | 11/12/2023 | |
7.2.2 | 3,072 | 11/10/2023 | |
7.2.1 | 507 | 11/9/2023 | |
7.2.0 | 3,689 | 11/9/2023 | |
7.1.0 | 833 | 11/7/2023 | |
7.0.10 | 7,520 | 10/7/2023 | |
7.0.9 | 13,407 | 8/27/2023 | |
7.0.8 | 2,404 | 8/25/2023 | |
7.0.5 | 5,742 | 8/10/2023 | |
7.0.4 | 5,746 | 7/27/2023 | |
7.0.3 | 11,145 | 6/21/2023 | |
7.0.2 | 937 | 6/19/2023 | |
7.0.1 | 1,955 | 6/17/2023 | |
7.0.0 | 902 | 6/17/2023 | |
6.8.7 | 9,574 | 5/21/2023 | |
6.8.6 | 677 | 5/19/2023 | |
6.8.5 | 662 | 5/19/2023 | |
6.8.3 | 1,461 | 5/16/2023 | |
6.8.2 | 736 | 5/15/2023 | |
6.8.1 | 2,720 | 5/7/2023 | |
6.8.0 | 4,489 | 4/30/2023 | |
6.7.4 | 1,102 | 4/27/2023 | |
6.7.3 | 1,201 | 4/26/2023 | |
6.7.2 | 1,359 | 4/23/2023 | |
6.7.1 | 8,607 | 4/13/2023 | |
6.7.0 | 1,966 | 4/10/2023 | |
6.6.0 | 64,115 | 4/4/2023 | |
6.5.3 | 2,688 | 3/29/2023 | |
6.5.2 | 802 | 3/29/2023 | |
6.5.1 | 1,339 | 3/27/2023 | |
6.5.0 | 2,565 | 3/26/2023 | |
6.4.3 | 715 | 3/26/2023 | |
6.4.2 | 1,203 | 3/26/2023 | |
6.4.1 | 2,834 | 3/24/2023 | |
6.4.0 | 879 | 3/23/2023 | |
6.3.2 | 2,889 | 3/22/2023 | |
6.3.1 | 5,788 | 3/17/2023 | |
6.3.0 | 1,673 | 3/17/2023 | |
6.2.0 | 776 | 3/16/2023 | |
6.1.0 | 3,326 | 3/14/2023 | |
6.0.1 | 1,801 | 3/12/2023 | |
6.0.0 | 1,108 | 3/11/2023 | |
5.1.2 | 780 | 3/10/2023 | |
5.1.1 | 801 | 3/9/2023 | |
5.1.0 | 1,133 | 3/8/2023 | |
5.0.2 | 1,619 | 3/6/2023 | |
5.0.1 | 1,345 | 3/2/2023 | |
5.0.0 | 971 | 3/2/2023 | |
4.4.4 | 2,417 | 2/18/2023 | |
4.4.3 | 1,846 | 2/10/2023 | |
4.4.2 | 1,303 | 2/7/2023 | |
4.4.1 | 974 | 2/4/2023 | |
4.4.0 | 750 | 2/4/2023 | |
4.3.0 | 988 | 1/31/2023 | |
4.2.0 | 946 | 1/28/2023 | |
4.1.0 | 851 | 1/27/2023 | |
4.0.2 | 1,199 | 1/20/2023 | |
4.0.1 | 1,043 | 1/17/2023 | |
4.0.0 | 2,343 | 1/9/2023 | |
3.0.1 | 8,303 | 4/14/2022 | |
3.0.0 | 2,400 | 6/20/2021 | |
2.0.1 | 956 | 5/29/2021 | |
2.0.0 | 1,027 | 5/29/2021 | |
1.0.1 | 969 | 5/2/2021 | |
1.0.0 | 1,153 | 5/1/2021 |
Bump version to 6.0.0
- Added support for Azure OpenAI
Bump version to 5.1.2
- Fixed an issue when deleting personal account fine tuned models
Bump version to 5.1.1
- Refactored Model validation
- Added additional default models
- Deprecate OpenAIClient.DefaultModel
- Implemented chat completion streaming
- Refactored immutable types
Bump version to 5.1.0
- Add support for Audio endpoint and Whisper models
- Audio Speech to text
- Audio translation
Bump version to 5.0.2
- Support multiple inputs in embedding
- Added better model validation in all endpoints
Bump version to 5.0.1
- Fixed chat parameters
Bump version to 5.0.0
- Added Chat endpoint
Bump version to 4.4.4
- ImageEditRequest mask is now optional so long as texture has alpha transparency
- ImageVariationRequest added constructor overload for memory stream image
- Updated AuthInfo parameter validation
- Renamed OPEN_AI_ORGANIZATION_ID -> OPENAI_ORGANIZATION_ID
Bump version to 4.4.3
- added `OPEN_AI_ORGANIZATION_ID` environment variable
- deprecated `Organization` use `OrganizationId` instead
Bump version to 4.4.2
- Removed a useless assert
- Updated docs
Bump version to 4.4.1
- hotfix to CompletionsEndpoint to use IEnumerable<string>
- hotfix to cleanup Images endpoints
Bump version to 4.4.0
- Renamed Choice.Logprobs -> Choice.LogProbabilities
- Renamed OpenAI.Completions.Logprobs -> OpenAI.Completions.OpenAI.Completions
- Renamed CompletionRequest parameter names:
- max_tokens -> maxTokens
- top_p -> topP
- Updated CompletionRequest to accept IEnumerable<string> values for prompts and stopSequences
- Refactored all endpoints to use new response validation extension
- Added `CancellationToken` to most endpoints that had long running operations