Many edge computing and IoT applications require adaptive and on-line learning architectures for fast and low-power processing of locally sensed signals. A promising class of architectures to solve this problem is that of in-memory computing ones, based on event-based hybrid memristive-CMOS devices. In this work, we present an example of such systems that supports always-on on-line learning. To overcome the problems of variability and limited resolution of ReRAM memristive devices used to store synaptic weights, we propose to use only their High Conductive State (HCS) and control their desired conductance by modulating their programming Compliance Current (I CC ). We describe the spike-based learning CMOS circuits that are used to modulate the synaptic weights and demonstrate the relationship between the synaptic weight, the device conductance, and the I CC used to set its weight, with experimental measurements from a 4kb array of HfO 2 -based devices. To validate the approach and the circuits presented, we present circuit simulation results for a standard CMOS 180nm process and system-level behavioral simulations for classifying hand-written digits from the MNIST data-set with classification accuracy of 92.68% on the test set.