Frontal midline (fm-)theta activity has been related to working memory (WM) processes, as it typically increases with WM load. The robustness of this effect, however, varies across studies and subjects, putting limits to its interpretation. We hypothesized that variation in the fm-theta effect may reflect individual differences in task difficulty with increasing WM load as indicated by behavioural responses. We further tested whether effects in the alpha range are robust markers of WM load. We recorded 64-channel EEG from 24 healthy adults while they memorized either 2 or 4 unfamiliar symbols (low vs. high WM load) in a modified Sternberg task. The last 2 s of the retention phase were analyzed for WM load-related changes in the theta (5-7 Hz) and alpha range (lower: 8-10 Hz, upper: 10.5-12.5 Hz). Higher WM load led to less accurate and slower responses. The increase of fm-theta with WM load was most pronounced at fm electrodes, localized to anterior cingulate regions, and correlated with the participants' decrease in accuracy due to higher WM load. Alpha peak frequency increased in the high compared to the low WM load condition, corresponding to a decrease in lower alpha range across all channels. The results demonstrate that previously reported variation in fm-theta workload effects can partly be explained by variation in task difficulty indexed by individual task accuracy. Moreover, the results also demonstrate that alpha WM load effects are prominent when separating upper and lower alpha.