Csv read thru gemini api

General
1

Hello Team, I am facing the below error and following the phidata github article. Please let me know, if any challenges.

from phi.agent import Agent
from phi.knowledge.csv import CSVUrlKnowledgeBase
from phi.vectordb.pgvector import PgVector
from phi.model.google import Gemini

db_url = "postgresql+psycopg://ai:ai@localhost:5432/ai"

from dotenv import find_dotenv
from dotenv import load_dotenv
env_file = find_dotenv(".env")
load_dotenv(env_file)

knowledge_base = CSVUrlKnowledgeBase(
    urls=["https://phi-public.s3.amazonaws.com/csvs/employees.csv"],
    vector_db=PgVector(table_name="csv_documents", db_url=db_url),
)
knowledge_base.load(recreate=False)  # Comment out after first run

agent = Agent(
    model=Gemini(id="gemini-1.5-flash"),
    knowledge_base=knowledge_base,
    search_knowledge=True,
)

agent.print_response("What is the average salary of employees in the Marketing department?", markdown=True)

Error:

/Users/467501/Desktop/GenAI-POC/Phidata/venv/bin/python /Users/467501/Desktop/GenAI-POC/Phidata/csv_url.py 
INFO     Creating collection                                                    
INFO     Loading knowledge base                                                 
INFO     Reading: https://phi-public.s3.amazonaws.com/csvs/employees.csv        
INFO     Reading uploaded file: employees.csv                                   
ERROR    Error processing document 'employees': Error code: 429 - {'error':     
         {'message': 'You exceeded your current quota, please check your plan   
         and billing details. For more information on this error, read the docs:
         https://platform.openai.com/docs/guides/error-codes/api-errors.',      
         'type': 'insufficient_quota', 'param': None, 'code':                   
         'insufficient_quota'}}                                                 
ERROR    Error processing document 'employees': Error code: 429 - {'error':     
         {'message': 'You exceeded your current quota, please check your plan   
         and billing details. For more information on this error, read the docs:
         https://platform.openai.com/docs/guides/error-codes/api-errors.',      
         'type': 'insufficient_quota', 'param': None, 'code':                   
         'insufficient_quota'}}                                                 
/Users/467501/Desktop/GenAI-POC/Phidata/venv/lib/python3.9/site-packages/phi/vectordb/pgvector/pgvector.py:320: SAWarning: Column 'ai.csv_documents.id' is marked as a member of the primary key for table 'ai.csv_documents', but has no Python-side or server-side default generator indicated, nor does it indicate 'autoincrement=True' or 'nullable=True', and no explicit value is passed.  Primary key columns typically may not store NULL.
  sess.execute(insert_stmt, batch_records)
ERROR    Error with batch starting at index 0: (psycopg.errors.NotNullViolation)
         null value in column "id" of relation "csv_documents" violates not-null
         constraint                                                             
         DETAIL:  Failing row contains (null, null, {}, {}, null, null, null,   
         2025-01-09 08:43:05.968246-05, null, null).                            
         [SQL: INSERT INTO ai.csv_documents DEFAULT VALUES]                     
         (Background on this error at: https://sqlalche.me/e/20/gkpj)           
ERROR    Error inserting documents: (psycopg.errors.NotNullViolation) null value
         in column "id" of relation "csv_documents" violates not-null constraint
         DETAIL:  Failing row contains (null, null, {}, {}, null, null, null,   
         2025-01-09 08:43:05.968246-05, null, null).                            
         [SQL: INSERT INTO ai.csv_documents DEFAULT VALUES]                     
         (Background on this error at: https://sqlalche.me/e/20/gkpj)           
Traceback (most recent call last):
  File "/Users/467501/Desktop/GenAI-POC/Phidata/venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
    self.dialect.do_execute(
  File "/Users/467501/Desktop/GenAI-POC/Phidata/venv/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 941, in do_execute
    cursor.execute(statement, parameters)
  File "/Users/467501/Desktop/GenAI-POC/Phidata/venv/lib/python3.9/site-packages/psycopg/cursor.py", line 97, in execute
    raise ex.with_traceback(None)
psycopg.errors.NotNullViolation: null value in column "id" of relation "csv_documents" violates not-null constraint
DETAIL:  Failing row contains (null, null, {}, {}, null, null, null, 2025-01-09 08:43:05.968246-05, null, null).

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/467501/Desktop/GenAI-POC/Phidata/csv_url.py", line 17, in <module>
    knowledge_base.load(recreate=False)  # Comment out after first run
  File "/Users/467501/Desktop/GenAI-POC/Phidata/venv/lib/python3.9/site-packages/phi/knowledge/agent.py", line 104, in load
    self.vector_db.insert(documents=documents_to_load, filters=filters)
  File "/Users/467501/Desktop/GenAI-POC/Phidata/venv/lib/python3.9/site-packages/phi/vectordb/pgvector/pgvector.py", line 320, in insert
    sess.execute(insert_stmt, batch_records)
  File "/Users/467501/Desktop/GenAI-POC/Phidata/venv/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 2362, in execute
    return self._execute_internal(
  File "/Users/467501/Desktop/GenAI-POC/Phidata/venv/lib/python3.9/site-packages/sqlalchemy/orm/session.py", line 2256, in _execute_internal
    result = conn.execute(
  File "/Users/467501/Desktop/GenAI-POC/Phidata/venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1418, in execute
    return meth(
  File "/Users/467501/Desktop/GenAI-POC/Phidata/venv/lib/python3.9/site-packages/sqlalchemy/sql/elements.py", line 515, in _execute_on_connection
    return connection._execute_clauseelement(
  File "/Users/467501/Desktop/GenAI-POC/Phidata/venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1640, in _execute_clauseelement
    ret = self._execute_context(
  File "/Users/467501/Desktop/GenAI-POC/Phidata/venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
    return self._exec_single_context(
  File "/Users/467501/Desktop/GenAI-POC/Phidata/venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context
    self._handle_dbapi_exception(
  File "/Users/467501/Desktop/GenAI-POC/Phidata/venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 2355, in _handle_dbapi_exception
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "/Users/467501/Desktop/GenAI-POC/Phidata/venv/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context
    self.dialect.do_execute(
  File "/Users/467501/Desktop/GenAI-POC/Phidata/venv/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 941, in do_execute
    cursor.execute(statement, parameters)
  File "/Users/467501/Desktop/GenAI-POC/Phidata/venv/lib/python3.9/site-packages/psycopg/cursor.py", line 97, in execute
    raise ex.with_traceback(None)
sqlalchemy.exc.IntegrityError: (psycopg.errors.NotNullViolation) null value in column "id" of relation "csv_documents" violates not-null constraint
DETAIL:  Failing row contains (null, null, {}, {}, null, null, null, 2025-01-09 08:43:05.968246-05, null, null).
[SQL: INSERT INTO ai.csv_documents DEFAULT VALUES]
(Background on this error at: https://sqlalche.me/e/20/gkpj)

Process finished with exit code 1
2

Hey @karthidec this seems like an error from openai. You’re getting error 429 because either you used all your free tokens or 3 months have passed since you signed up. Please check your billing information on openai

3

Yes, that is my query … I don’t want to use openai because my company is restricting it. I am using gemini as alternative.

Phidata is tightly coupled to openai? I’m not sure, from where it is referring now? I have removed all openai references in my above code.

I am trying to execute the codes without any openai references and fully through gemini and google flavored. Every time, openai is turning up … like to know the community team thoughts on this.

4

If no embedder is specified in vector db, the system defaults to using OpenAI. To utilize the GeminiEmbedder, you can import from phi.embedder.google import GeminiEmbedder

please refer to the documentation for more: Gemini Embedder - Phidata