Skip to content
article

volatile is not a lock: 4 ISR bugs I have personally shipped to pump controllers

Works in debug build, hangs in release. Every embedded engineer meets these four

Karthikeyan S@karthik_embedded

September 7, 2026 · 3 min read

All four of these went to the field in products with my name in the git blame. All four passed bench testing. Sharing so you can ship different bugs.

Bug 1: the missing volatile

bool adc_done = false;               /* BUG: not volatile */

void ADC_IRQHandler(void) {
    adc_value = ADC1->DR;
    adc_done = true;
}

void read_adc(void) {
    start_conversion();
    while (!adc_done) { }            /* at -O2 this becomes while(1) */
    adc_done = false;
}

The compiler sees nothing in the loop changing adc_done, so it reads it once and loops forever. Debug build (-O0) works, release hangs. Fix: volatile bool adc_done. This one everybody knows. The next three are why I am writing this.

Bug 2: volatile, but read-modify-write

volatile uint32_t pulse_count;

void EXTI0_IRQHandler(void) { pulse_count++; }

void every_second(void) {
    flow = pulse_count * K;
    pulse_count = 0;                 /* BUG: a pulse can arrive between these two lines */
}

volatile makes every access real. It does not make two accesses one. If the interrupt fires between the read and the write of zero, that pulse is lost. Our flow meter under-read by ~0.3% at high flow. Customer noticed in the water bill, not us. Fix:

uint32_t primask = __get_PRIMASK();
__disable_irq();
uint32_t n = pulse_count;
pulse_count = 0;
__set_PRIMASK(primask);

Save and restore PRIMASK, don't blindly call __enable_irq(), because you might already be inside someone else's critical section and you just enabled interrupts for them.

Bug 3: the 49.7 day bug

volatile uint64_t ms_ticks;          /* incremented in SysTick */

uint64_t now = ms_ticks;             /* BUG on a 32-bit MCU */

On Cortex-M a 64-bit read is two 32-bit loads. If SysTick fires between them exactly when the low word rolls over from 0xFFFFFFFF to 0, you read the old high word with the new low word (or the reverse), and time jumps by 2^32 ms. That is 49.7 days. Our maintenance scheduler thought 49 days passed in one millisecond and raised "service overdue" on a brand new pump. Took months to see it twice. Fix, either disable interrupts around the read, or the read-twice trick:

uint64_t get_ticks(void) {
    uint32_t hi, lo;
    do {
        hi = ticks_hi;
        lo = ticks_lo;
    } while (hi != ticks_hi);        /* high word changed, read again */
    return ((uint64_t)hi << 32) | lo;
}

Bug 4: volatile does not order your other memory

uint8_t tx_buf[64];
volatile bool tx_ready;

fill(tx_buf);
tx_ready = true;                     /* DMA ISR starts sending when it sees this */

Volatile accesses are kept in order with each other. Normal accesses (the buffer) can be moved around them by the compiler. And on a Cortex-M7 with data cache (STM32H7), the buffer can still be sitting in cache when DMA reads RAM. You send old data. Fix: a compiler and hardware barrier, and on M7 clean the cache for that buffer before starting DMA:

fill(tx_buf);
SCB_CleanDCache_by_Addr((uint32_t *)tx_buf, sizeof tx_buf);
__DMB();
tx_ready = true;

(For the cache clean, the buffer should be 32-byte aligned and sized, or you clean your neighbour's data too.)

What volatile actually means

"Every access to this variable really happens, in order relative to other volatile accesses." That's all. Not atomic. Not a lock. Not a memory barrier for other variables. The Linux kernel people have a famous document on this: volatile considered harmful. Different world, same lesson.

Keep ISRs short, share as little as possible, and when you share, think "what if the interrupt fires between these two lines". It will. In field only.

Written by

Karthikeyan S

Embedded engineer, Coimbatore. STM32, ESP32, motor controllers for pumps (Kovai is pump city da). C, RTOS, oscilloscope, filter coffee. எம்பெடட் தான் உண்மையான கோடிங் 😄

Get the best of #bare_metal weekly

One email each Monday with the most useful posts. No spam; unsubscribe in one click.

Comments

8

Sign in to join the conversation.

Sign in to comment
Mehmet Aydoğan@malloc_mehmet#4 in Bare Metal
Correct, all four. Bug 3 I saw in a PLC in 2008, same 49.7 days, customer in Germany, three engineers and two months. Good post. No framework, no hype, only the truth.
1 net votes
Daan Visser@daan_fpga#8 in Bare Metal
Bug 4 has a cousin in FPGA land: the CPU writes a descriptor, then sets the "go" bit in a register, and the fabric reads the descriptor before the write left the CPU. Same fix, a barrier. Very nice post.
3 net votes
Matthias Kühn@m_kuehn#6 in Bare Metal
MISRA and our static analyser have opinions about every one of these, and they are right every time. Bug 4 is the one that survives code review, because the code looks correct.
1 net votes
Ondřej Hrubý@rewrite_it_in_rust#7 in Bare Metal
C programmers in 2026 still shipping data races to water pumps lmao. in Rust bug 1, 2, 3 are compile error. imagine choosing this 🦀
-9 net votes
Karthik Subramaniam@karthi_rustacean#8 in Bare Metal
Ondřej, bug 3 compiles fine in Rust with static mut and an unsafe block, which is exactly what half the embedded Rust examples did for years. And bug 4 is cache coherence with DMA. No language fixes that, it is hardware. Rust helps with 1 and 2, and I say that as someone who writes Rust every day.
1 net votes
Karthikeyan S@karthik_embedded#1 in Bare Metal
Thank you Karthik. Same name, same state, opposite language, same conclusion 😄
1 net votes
Frank Ostrowski@frank_cpp981 rep in Bare Metal
For the C++ people about to say "just use std::atomic<uint64_t>": on Cortex-M3 and M4 that is not lock free. Check is_always_lock_free. It quietly calls a library function that disables interrupts, which may or may not be what you want inside an ISR. Regards.
1 net votes
Diego Ramírez@diego_firmware#3 in Bare Metal
The PRIMASK save and restore tip. I have seen so many codebases with __enable_irq() at the end of every critical section, and then a nested one enables interrupts in the middle of the outer one. Great post
5 net votes