Skip to content

Tags

#data-science

Posts tagged #data-science on Dynamo Geeks: 12 posts from the community, most useful first.

12 posts

Posts tagged #data-science

  1. Question

    Is it okay to use ChatGPT for a take-home assignment in a grad data science interview?

    Hi everyone, I am finishing Master of Data Science and applying for grad roles. I got a take-home assignment from one company: analyse a customer churn dataset, build a model, write short report, 5 days. The instructions don't say anything about AI tools. I have question: Is it okay to use ChatGPT or Copilot to help? For example for pandas syntax, or to check the English in my report (my English is not native).…

    Zhang Xinyi 张欣怡14likes7comments

  3. Question

    Pandera, Great Expectations or plain asserts for data validation in a small team?

    Hola! Small data team (3 people), Python, pandas, some dbt, Airflow. We have zero data validation, and last month a supplier changed a column from euros to cents without telling anybody. Our revenue dashboard showed 100x for two days 🙃 Nobody noticed until the CFO was very happy, and then very angry. I want to add validation at the ingestion step. The options I see: pandera : schemas in Python, looks light. Great…

    Núria Puig13likes6comments

  4. Article

    the AI wrote me a running total that was 'almost right'. almost right is the worst kind

    ok quick one because this bit me today and i want to save someone an afternoon. asked an ai assistant for a running total of revenue. it gave me this, which looks totally fine: SELECT order_date, amount, SUM(amount) OVER (ORDER BY order_date) AS running_total FROM orders; ran it, numbers looked plausible, almost shipped it to the dashboard. then i noticed the running total jumped by the same amount on every row with…

    Mo Siddiqui13likes7comments

  5. Article

    We moved our nightly jobs to the greenest hours of the grid. Here's what it saved (spoiler: less than you'd think, and that's the point)

    I'm a data analyst at a community energy co-op in Bristol. We're small: about 400 member households, a few solar roofs, a share in a wind turbine, and one Linux box in a rack in a shared data centre that runs all our nightly jobs (meter data imports, billing calcs, the dashboards members look at). People in this topic keep asking about carbon-aware scheduling, so here's an honest write-up of doing it for real,…

    Freya Collins41likes17comments

  6. Question

    boss wants 'chat with your data' by Q4. anyone actually running LLM-to-SQL for business users?

    hi all. data analyst kat a bank in KL. my boss went to a vendor event and now he wants "chat with your data" for the business users by Q4 lah. type a question in English or Malay, get the answer from our warehouse. sounds very nice on the slide. my worries, maybe I am overthinking: our schema has 600+ tables, column names like TXN_AMT_2 and CUST_FLG_X . even humans don't know what they mean. "revenue" means 3…

    Nurul Aisyah8likes8comments

  7. Article

    Teu modelo tem 99% de acurácia? Bah, então me mostra a matriz de confusão

    Bah, tchê. Toda semana aparece alguém no LinkedIn com "meu modelo atingiu 99% de acurácia" e um gráfico bonito. E toda semana eu tenho as mesmas três perguntas. Vou deixar aqui pra poder mandar o link 🧉 1. Qual é a proporção das classes? Se 99% das transações não são fraude, um modelo que responde "não é fraude" pra tudo tem 99% de acurácia. E não pega fraude nenhuma. Tu viu? Acurácia sozinha não diz nada quando as…

    Bruna Schmitt6likes7comments

  8. Article

    How I check a strike location with free satellite data, and when I refuse to publish

    A reader asked how we decide whether a claimed strike location holds up. Here's the free toolkit and the actual decision rule. Nothing here needs a paid account. The free sources Sentinel-2 (ESA, optical): 10 m resolution in the visible bands, a new image roughly every 5 days at mid latitudes. Big burn scars, collapsed large buildings, craters in fields. Not cars, not people. Sentinel-1 (radar): sees through clouds…

    Naomi Feld9likes6comments

  9. Question

    Engineers who have worked on UPI: why do failed transactions take days to reverse when the rule says T+1?

    Working on a data story about failed UPI transactions. Readers sent us about 1,800 complaints (voluntarily, everything anonymised) where money left their account but never reached the merchant or the person they were paying. The RBI's 2019 circular on turnaround times says, as I read it, that a failed UPI transaction should be auto-reversed by T+1, and after that the bank owes the customer ₹100 for every day of…

    Ira Saxena11likes12comments

  10. Article

    Antes de culpar al modelo, mira tus etiquetas

    Lo prometí en el otro post, así que aquí va en castellano (y algo de català, perdón). Teníamos un clasificador de reseñas de clientes con 5 categorías: producto, envío, precio, atención al cliente y "otro". Llevábamos dos meses probando modelos: TF-IDF con regresión logística, un BERT multilingüe afinado y un LLM con few-shot. Los tres se quedaban en 0.78-0.80 de F1. Siempre lo mismo. Sospechoso, ¿no? El experimento…

    Núria Puig7likes7comments

  11. Article

    Scraping county election results on election night without getting blocked (or getting it wrong)

    Primary season means I spend a lot of evenings refreshing county results pages. Here's how our desk does it, after some years and some embarrassing mistakes (mine). 1. Call the county first Seriously. Many counties publish a JSON or CSV feed behind the pretty results page, and many use the same few results vendors, so once you know one feed format you know a lot of them. Ask the elections office what the official…

    Naomi Feld31likes10comments

  12. Article

    Your model didn't get better, your test set leaked (a checklist)

    A ver. Three times this year a model "improved" by a lot, and three times it was leakage. The last one was a fine-tuned classifier that jumped from 0.81 to 0.93 F1, and my manager already had a slide 🙃 This is the checklist I run now before I believe anything. 1. Exact and near duplicates between train and test The 80% of my leakage problems are this. Customer reviews, support tickets, product descriptions: people…

    Núria Puig19likes7comments

Get the best of Dynamo Geeks weekly

One email each Monday with the most useful posts. No spam; unsubscribe in one click.