I decided to try AI.
I devised a simple task: create a spreadsheet to calculate loan payments. Specifically for an auto loan.
For the AI engine, I selected Google's Gemini. This was a random choice; I had no reason to favor Gemini over other AI engines.
My prompt to Gemini was something along the lines of "create a spreadsheet to calculate payments for a car loan". I don't remember the exact prompt.
I had expected Gemini to create a spreadsheet that let me enter the cost of the car, a down payment, an interest rate, and a term. Google did create a spreadsheet (after I went to various setting pages and enabled "smart features") but the spreadsheet ... didn't work.
The spreadsheet was kind of correct but mostly wrong. There were two types of errors, both related to formulas. The first was an error in column references: the formulas referred to column A, when the values to which they referred were in column B.
The second error was one that occurred when transferring the AI results into the spreadsheet. One formula used the 'PMT' function with three arguments. The three arguments were split into three different cells. The splits occurred where there are normally commas. So the proper formula 'PMT(A4/12,A5,-A6)' became the three cells 'PMT(A4/12', 'A5, and '-A6)'. Notice that the splits are exactly where the commas should be. (The negative sign in the last argument is correct; that's the way the PMT function works.)
Upon seeing the spreadsheet (and its errors), my first thought was "If this is the quality of Gemini, Google is in big trouble.".
My second thought was "Maybe I should write another prompt, asking Gemini to fix the spreadsheet.". I expect many people would do that. I chose not to. I was disappointed in Gemini's output: these two problems are significant. How could Gemini make these errors?
If AI needs additional prompts to get things right, even for a trivial task, what do we do for larger and more complex tasks? I can re-prompt Gemini to get it to fix errors, but only if I know that there are errors.
I don't see how AI can help if it generates the wrong result and needs additional prompting to make things work. And I'm not sure that I will be able to identify errors or problems when the thing created by AI is large and complex. Asking AI to check its work requires that we trust AI, which we cannot. Even asking a different AI engine isn't a solution; how can we trust that engine?
My impression of AI is that it can do some interesting things but we cannot trust the results. If we cannot trust the results then I'm not sure that AI helps -- in any effort. At least for now.
