Learning Hidden Markov Model of Stochastic Environment with Bio-inspired Probabilistic Temporal Memory


Panov A. Kuderov P.


Learning models online in partially observable stochastic environments can still be challenging for artificial intelligent agents. In this paper, we propose an algorithm for the probabilistic modeling of observation sequences based on the neurophysiological model of the human cortex, which is notoriously fit for this task. We argue that each dendritic segment of a pyramidal neuron may be considered an independent naive Bayesian detector of afferent neuron activity patterns. Experiments show that our model can learn the dynamics of the partially observable environments for very few interactions online and reliably predict probabilistic distributions of observations for several future time steps using Monte Carlo sampling. Additionally, we compare our algorithm with a biologically inspired HMM implementation of temporal memory and standard LSTM on both Markov chain-generated character sequences and observation image sequences in a pinball-like environment.

External links

DOI: 10.1007/978-3-031-50381-8_33

ResearchGate: https://www.researchgate.net/publication/378200007_Learning_Hidden_Markov_Model_of_Stochastic_Environment_with_Bio-inspired_Probabilistic_Temporal_Memory

Reference link

Dzhivelikian, E., Kuderov, P., Panov, A.I. (2024). Learning Hidden Markov Model of Stochastic Environment with Bio-inspired Probabilistic Temporal Memory // In: Samsonovich, A.V., Liu, T. (eds) Biologically Inspired Cognitive Architectures 2023. BICA 2023. Studies in Computational Intelligence, vol 1130, pp. 330–339.