A robust way to unconver the focus of visual attention from (simulated) noisy eye tracking data provided by the hidden Markov model was discussed. It was found that a hidden simi-Markov model (HSMM) with explicit state duration PDF representing task-constrained visual attention was more stable and accurate to represent visual attention duration. HSMM used an additional Gaussian component to the observation distribution PDF with larger standard deviation to ensure less differentiation between eye movement positions for away from the object. Analysis shows that HMM and HSMM performed better in terms of accuracy and instability than the baseline non-HMM method.