Posts tagged #debugging on Dynamo Geeks: 3 posts from the community, most useful first.
After fixing our retrieval (thanks for the help in my other post), now I see a different problem. In maybe 10% of the failed eval cases, the correct chunk IS in the context, I check it, but the model still answer wrong. Example: chunk say return window is 7 days for electronics and 30 days for clothes. Customer ask about headphones, bot say 30 days. Our context has 5 chunks, about 3000 tokens total, the correct one…
promised this one ages ago so here it is. We had an end to end test for policy renewals. It ran every night at half twelve: create a policy starting today, jump forward a year, check the renewal quote and the end date. Green for months. Then one night at the end of October it went red, a policy that started on the 26th came out with an end date of the 25th the following year. Next night green again. Classic flake,…
This was some years ago but I still think about it every winter. We had a body control module, nothing special, talking on CAN at 500 kbit/s with the other modules. On the test bench everything perfect. In the field, some customers report that the module "die" in the morning and come back after the car is driving some minutes. Only in the north. Only in winter. First we suspect the power supply, because cold…
