Increasing Max Tokens Limit for Azure OpenAI LLM Model in PhiData

General
1

I am working on enhancing the capabilities of my agent powered by Azure OpenAI in PhiData. One of the key improvements I’m aiming for is increasing the max_tokens limit to allow for processing longer inputs and generating more comprehensive responses. Here’s how I’ve set up the Azure OpenAI model in my code:

azure_model: AzureOpenAIChat = AzureOpenAIChat(
    id=os.getenv("AZURE_OPENAI_MODEL_NAME"),
    api_key=os.getenv("AZURE_OPENAI_API_KEY"),
    azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
    azure_deployment=os.getenv("AZURE_OPENAI_DEPLOYMENT"),
)

Could anyone provide guidance or best practices on how to modify or configure the max_tokens parameter in PhiData when using Azure OpenAI’s LLM? Any suggestions on optimizing token usage for better performance would be greatly appreciated!

2

Hello @atharvx !

AzureOpenAIChat inherits from OpenAILike which inherits from OpenAIChat class.

So, you can utilize the max_tokens param of OpenAIChat in your Azure implementation to limit the max amount of tokens. Do let us know if you have any questions.