Optimal segmentation of pupillometric images for estimating pupil shape parameters

Pupil Size Prediction Techniques Based on Convolution Neural Network

The size of one’s pupil can indicate one’s physical condition and mental state. When we search related papers about AI and the pupil, most studies focused on eye-tracking. This paper proposes an algorithm that can calculate pupil size based on a convolution neural network (CNN). Usually, the shape of the pupil is not round, and 50% of pupils can be calculated using ellipses as the best fitting shapes. This paper uses the major and minor axes of an ellipse to represent the size of pupils and uses the two parameters as the output of the network. Regarding the input of the network, the dataset is in video format (continuous frames). Taking each frame from the videos and using these to train the CNN model may cause overfitting since the images are too similar. This study used data augmentation and calculated the structural similarity to ensure that the images had a certain degree of difference to avoid this problem. For optimizing the network structure, this study compared the mean error with changes in the depth of the network and the field of view (FOV) of the convolution filter. The result shows that both deepening the network and widening the FOV of the convolution filter can reduce the mean error. According to the results, the mean error of the pupil length is 5.437% and the pupil area is 10.57%. It can operate in low-cost mobile embedded systems at 35 frames per second, demonstrating that low-cost designs can be used for pupil size prediction.

Efficient and robust pupil size and blink estimation from near-field video sequences for human-machine interaction

Monitoring pupil and blink dynamics has applications in cognitive load measurement during human-machine interaction. However, accurate, efficient, and robust pupil size and blink estimation pose significant challenges to the efficacy of real-time applications due to the variability of eye images, hence to date, require manual intervention for fine tuning of parameters. In this paper, a novel self-tuning threshold method, which is applicable to any infrared-illuminated eye images without a tuning parameter, is proposed for segmenting the pupil from the background images recorded by a low cost webcam placed near the eye. A convex hull and a dual-ellipse fitting method are also proposed to select pupil boundary points and to detect the eyelid occlusion state. Experimental results on a realistic video dataset show that the measurement accuracy using the proposed methods is higher than that of widely used manually tuned parameter methods or fixed parameter methods. Importantly, it demonstrates convenience and robustness for an accurate and fast estimate of eye activity in the presence of variations due to different users, task types, load, and environments. Cognitive load measurement in human-machine interaction can benefit from this computationally efficient implementation without requiring a threshold calibration beforehand. Thus, one can envisage a mini IR camera embedded in a lightweight glasses frame, like Google Glass, for convenient applications of real-time adaptive aiding and task management in the future.

System and measurement method for binocular pupillometry to study pupil size variability

BACKGROUND

An objective and noninvasive examination of pupil size variability can be used to assess the activity of the autonomous nervous system. We designed a system that enables binocular, fast, and accurate recordings of different types of pupil variabilities, which are synchronous with other biosignals. This type of measurement system is needed to extend the scope of pupillometry applications.

METHODS

In the proposed system, the left and right eyes are independently and interchangeably illuminated to generate alternating images, which are successively acquired by a single camera. The system is composed of four functional modules: the image acquisition module, the image processing unit, the light stimulator, and the controller. The proposed image processing algorithm approximates the shape of the pupil using the best-fit ellipse. The user control panel (controller) precisely sets the stimuli parameters and controls the entire measurement procedure.

RESULTS

The computer-based binocular system records the pupil size during the pupil light reflexes (direct and indirect) and spontaneous pupil size fluctuations, at a sampling rate up to 75 Hz, with a resolution better than 0.02 mm. Our initial laboratory tests confirmed that the new system is fast and precise (system accuracy better than 0.5% and repeatability better than 4%).

CONCLUSIONS

The proposed system's unique geometry and construction, and the method it uses to detect images from each eye, allows us to monitor the right and left eyes using a single camera with no overlap between the images. The system does not require a very experienced operator, because it is relatively simple and easy to use. Importantly, it is comfortable for the subjects. Additionally, the presented system can operate with other bio-measurement systems using a synchronous signal. These system capabilities can expand the scope of pupillometry research applications.

An open-source, FireWire camera-based, Labview-controlled image acquisition system for automated, dynamic pupillometry and blink detection

The dynamic, accurate measurement of pupil size is extremely valuable for studying a large number of neuronal functions and dysfunctions. Despite tremendous and well-documented progress in image processing techniques for estimating pupil parameters, comparatively little work has been reported on practical hardware issues involved in designing image acquisition systems for pupil analysis. Here, we describe and validate the basic features of such a system which is based on a relatively compact, off-the-shelf, low-cost FireWire digital camera. We successfully implemented two configurable modes of video record: a continuous mode and an event-triggered mode. The interoperability of the whole system is guaranteed by a set of modular software components hosted on a personal computer and written in Labview. An offline analysis suite of image processing algorithms for automatically estimating pupillary and eyelid parameters were assessed using data obtained in human subjects. Our benchmark results show that such measurements can be done in a temporally precise way at a sampling frequency of up to 120 Hz and with an estimated maximum spatial resolution of 0.03 mm. Our software is made available free of charge to the scientific community, allowing end users to either use the software as is or modify it to suit their own needs.

Robust real time eye tracking for computer interface for disabled people

Gaze is a natural input for a Human Computer Interface (HCI) for disabled people, who have of course an acute need for a communication system. An efficient eye tracking procedure is presented providing a non-invasive method for real time detection of a subject eyes in a sequence of frames captured by low cost equipment. The procedure can be easily adapted to any subject and is adequately insensitive to changing of the illumination. The eye identification is performed on a piece-wise constant approximation of the frames. It is based on a discrete level set formulation of the variational approach to the optimal segmentation problem. This yields a simplified version of the original data retaining all the information relevant to the application. Tracking is obtained by a fast update of the optimal segmentation between successive frames. No eye movement model is required being the procedure fast enough to obtain the current frame segmentation as one step update from the previous frame segmentation.