My Model Is Fine, The Data Is Not
Nobody has said anything yet. Be the first.
My Model Is Fine, The Data Is Not
Data engineering, ML pipelines, labelling and data quality
Data engineering, pipelines, labelling, drift, and the training set that quietly rotted.
19 members
8 posts
Created Aug 2026
Community rules
No scraped personal data
Do not post datasets built from people who did not agree to be in them.
Bring the detail
Versions, error text, what you already tried. "It does not work" is not a question anyone can answer.
No drive-by self promotion
Share your own work when it answers the thread. A link with no context is a advert, and it gets removed.
Top contributors
Discussions
- Primary
Showing 8 posts.
Posts in t/my-model-is-fine
Is it okay to use ChatGPT for a take-home assignment in a grad data science interview?
Hi everyone, I am finishing Master of Data Science and applying for grad roles. I got a take-home assignment from one company: analyse a customer churn dataset, build a model, write short report, 5 days. The instructions don't say anything about AI tools. I have question: Is it okay to use ChatGPT or Copilot to help? For example for pandas syntax, or to check the English in my report (my English is not native).…
Accepted answer
hi xinyi! my honest view as someone who reviews take-homes at my bank: 1. using it for syntax and checking your English is fine lah. everyone googles pandas syntax, AI is the new google for that. 2. yes, mention it, one line at the end: 'I used ChatGPT for syntax lookups and proofreading. All analysis decisions and code are my own and I can explain them.' this reads as honest and mature, not weak. 3. this one is the important one. don't submit any line you can't explain. before you submit, go through every cell and say out loud why it is there. if you can't, rewrite it your way. what we actually look for: did you check for leakage, did you handle the class imbalance (churn is always imbalanced), did you explain the business meaning. AI won't think about those for you. you can do this! 💪
How are you versioning prompts together with their eval results?
A practical question rather than an opinion, for once. We have about 30 prompts in production across six features, and each has an eval set. Today the prompts live in the application repo as text files, eval results go into a spreadsheet, and the link between "this prompt version" and "these scores" is a commit hash someone pastes into a cell. It works until it does not: last week nobody could tell me which version…
- We are small so we do the git + CI way too, but one tip: store a hash of the eval set in the result file. We once 'improved' 4 points because someone removed the hard examples from the eval set. Bueno, it was me.
Airflow catchup ran 400 DAG runs at once and killed our warehouse, how to prevent this?
Deployed new DAG yesterday with start_date=datetime(2025, 9, 1) and schedule='@daily' . Did not set catchup. Airflow scheduled ~400 runs for all past days, they start in parallel and warehouse queue full for 2 hours, other team jobs stuck. Very embarrassing. with DAG( dag_id='orders_daily_snapshot', start_date=datetime(2025, 9, 1), schedule='@daily', ) as dag: ... I paused the DAG and marked the runs failed…
Why finance and marketing never agree on revenue (checklist from 6 months of pain)
Every month same meeting. Marketing dashboard say revenue X, finance report say Y, difference 4-8%, and data team (me) is asked to explain. After 6 months I made a checklist. Sharing in case it help someone. Where the numbers split Gross vs net. Marketing count order value at checkout. Finance count after cancellations, refunds and returns. Biggest gap is always this one. Which date. Order date, payment date,…
Pandera, Great Expectations or plain asserts for data validation in a small team?
Hola! Small data team (3 people), Python, pandas, some dbt, Airflow. We have zero data validation, and last month a supplier changed a column from euros to cents without telling anybody. Our revenue dashboard showed 100x for two days 🙃 Nobody noticed until the CFO was very happy, and then very angry. I want to add validation at the ingestion step. The options I see: pandera : schemas in Python, looks light. Great…
Accepted answer
Small team, I'd do two layers. 1. pandera at ingestion for schema and ranges. It's just Python, it reads like documentation and it fails loudly: import pandera as pa schema = pa.DataFrameSchema({ 'amount_eur': pa.Column(float, pa.Check.in_range(0, 50_000)), 'currency': pa.Column(str, pa.Check.isin(['EUR'])), }) schema.validate(df, lazy=True) 2. dbt tests (not_null, unique, accepted_values, plus a custom one) on the models you already have. For euros to cents specifically, a range check won't catch it if the max is generous. What catches it is a distribution check against history: today's median amount vs the last 30 days. If it moves 10x, fail. That's a 15-line custom test in dbt. GX can do this too, but for 3 people the setup cost is real.
the AI wrote me a running total that was 'almost right'. almost right is the worst kind
ok quick one because this bit me today and i want to save someone an afternoon. asked an ai assistant for a running total of revenue. it gave me this, which looks totally fine: SELECT order_date, amount, SUM(amount) OVER (ORDER BY order_date) AS running_total FROM orders; ran it, numbers looked plausible, almost shipped it to the dashboard. then i noticed the running total jumped by the same amount on every row with…
- Saving this for interviews. 'What is the default frame when you add ORDER BY to a window function?' separates the people who have been burned from the people who haven't.
dbt incremental model getting duplicate rows when late data arrive, how to fix?
Using dbt on BigQuery. Orders model is incremental, source table get updated when order status change (paid, shipped, refunded). Model look like this: {{ config(materialized='incremental') }} select order_id, status, amount, updated_at from {{ source('shop', 'orders') }} {% if is_incremental() %} where updated_at > (select max(updated_at) from {{ this }}) {% endif %} Problem 1: same order_id appear multiple times…
boss wants 'chat with your data' by Q4. anyone actually running LLM-to-SQL for business users?
hi all. data analyst kat a bank in KL. my boss went to a vendor event and now he wants "chat with your data" for the business users by Q4 lah. type a question in English or Malay, get the answer from our warehouse. sounds very nice on the slide. my worries, maybe I am overthinking: our schema has 600+ tables, column names like TXN_AMT_2 and CUST_FLG_X . even humans don't know what they mean. "revenue" means 3…
Accepted answer
We ran a pilot of exactly this in Singapore last year. Short answer: it works on a curated subset, never on the raw warehouse. What worked: we built a semantic layer first (about 40 business metrics in dbt, each with one agreed definition, plus descriptions for every column the model is allowed to see). The model writes queries against that layer, not against 600 tables. Accuracy on our test questions went from embarrassing to about 85% on the curated layer. What didn't: anything outside those 40 metrics. And the 15% wrong answers look exactly as confident as the right ones. So the real project is the semantic layer and the definitions. The chatbot is the last 10%.