Hi, I have data in several large CSV files, lots of rows and long entries in each column. Currently I am using the CsvTool to let my agent read it, but this is not performing very well, and eating up lots tokens, such that I reach Rate Limits quickly.
I suppose that I can convert the csv to txt files, and then load them as Text knowledge base. Is this the right way to go? If so, would it be an idea to have a KnowledgeBase type specific for CSVs?
Hey @LourensT, we have CSVKnowledgeBase that you can use. I am working on writing a cookbook and doc for the same but till then you can refer to the source code here
Hey @LourensT ! As @manthanguptaa pointed out, you can use CSVKnowledgeBase . But if your CSV document is text heavy then TextKnowledgeBase might work better. Do let us know if you have any further questions