Skip to content

t/my-model-is-fine

My Model Is Fine, The Data Is Not: top posts

The most useful posts in this topic, by likes from the community.

Posts

  1. Question

    Is it okay to use ChatGPT for a take-home assignment in a grad data science interview?

    Hi everyone, I am finishing Master of Data Science and applying for grad roles. I got a take-home assignment from one company: analyse a customer churn dataset, build a model, write short report, 5 days. The instructions don't say anything about AI tools. I have question: Is it okay to use ChatGPT or Copilot to help? For example for pandas syntax, or to check the English in my report (my English is not native).…

    Zhang Xinyi 张欣怡14likes7comments

  2. Question

    Pandera, Great Expectations or plain asserts for data validation in a small team?

    Hola! Small data team (3 people), Python, pandas, some dbt, Airflow. We have zero data validation, and last month a supplier changed a column from euros to cents without telling anybody. Our revenue dashboard showed 100x for two days 🙃 Nobody noticed until the CFO was very happy, and then very angry. I want to add validation at the ingestion step. The options I see: pandera : schemas in Python, looks light. Great…

    Núria Puig13likes6comments

  3. Article

    the AI wrote me a running total that was 'almost right'. almost right is the worst kind

    ok quick one because this bit me today and i want to save someone an afternoon. asked an ai assistant for a running total of revenue. it gave me this, which looks totally fine: SELECT order_date, amount, SUM(amount) OVER (ORDER BY order_date) AS running_total FROM orders; ran it, numbers looked plausible, almost shipped it to the dashboard. then i noticed the running total jumped by the same amount on every row with…

    Mo Siddiqui13likes7comments

  4. Question

    How are you versioning prompts together with their eval results?

    A practical question rather than an opinion, for once. We have about 30 prompts in production across six features, and each has an eval set. Today the prompts live in the application repo as text files, eval results go into a spreadsheet, and the link between "this prompt version" and "these scores" is a commit hash someone pastes into a cell. It works until it does not: last week nobody could tell me which version…

    Dana Kowalski10likes7comments

  5. Question

    boss wants 'chat with your data' by Q4. anyone actually running LLM-to-SQL for business users?

    hi all. data analyst kat a bank in KL. my boss went to a vendor event and now he wants "chat with your data" for the business users by Q4 lah. type a question in English or Malay, get the answer from our warehouse. sounds very nice on the slide. my worries, maybe I am overthinking: our schema has 600+ tables, column names like TXN_AMT_2 and CUST_FLG_X . even humans don't know what they mean. "revenue" means 3…

    Nurul Aisyah8likes8comments

  6. Article

    Why finance and marketing never agree on revenue (checklist from 6 months of pain)

    Every month same meeting. Marketing dashboard say revenue X, finance report say Y, difference 4-8%, and data team (me) is asked to explain. After 6 months I made a checklist. Sharing in case it help someone. Where the numbers split Gross vs net. Marketing count order value at checkout. Finance count after cancellations, refunds and returns. Biggest gap is always this one. Which date. Order date, payment date,…

    Lin Wei4likes3comments

  7. Question

    Airflow catchup ran 400 DAG runs at once and killed our warehouse, how to prevent this?

    Deployed new DAG yesterday with start_date=datetime(2025, 9, 1) and schedule='@daily' . Did not set catchup. Airflow scheduled ~400 runs for all past days, they start in parallel and warehouse queue full for 2 hours, other team jobs stuck. Very embarrassing. with DAG( dag_id='orders_daily_snapshot', start_date=datetime(2025, 9, 1), schedule='@daily', ) as dag: ... I paused the DAG and marked the runs failed…

    Lin Wei2likes3comments

  8. Question

    dbt incremental model getting duplicate rows when late data arrive, how to fix?

    Using dbt on BigQuery. Orders model is incremental, source table get updated when order status change (paid, shipped, refunded). Model look like this: {{ config(materialized='incremental') }} select order_id, status, amount, updated_at from {{ source('shop', 'orders') }} {% if is_incremental() %} where updated_at > (select max(updated_at) from {{ this }}) {% endif %} Problem 1: same order_id appear multiple times…

    Lin Wei2likes3comments

Get the best of Dynamo Geeks weekly

One email each Monday with the most useful posts. No spam; unsubscribe in one click.