Authors: F. J. González-Cañete aff001; J. L. López Rodríguez aff001; P. M. Galdón aff001; A. Díaz-Estrella aff001 Authors place of work: Department of Electronic Technology, University of Malaga, Campus de Teatinos, Malaga, Spain aff001 Published in the journal: PLoS ONE 14(11) Category: Research Article doi: https://doi.org/10.1371/journal.pone.0225053
We report the results of a study on the learnability of haptic icons used as alerts or notifications in smartphones. The aim was to explore the feasibility of using haptic icons to create assistive technologies for people with visual impairments. We compared the performance and satisfaction of users with different visual capacities (visually impaired vs. sighted) and using different learning processes (with or without a reinforcement learning stage). The reinforcement learning stage improves the recognition rate in both types of users, although the improvement obtained by the visually impaired users is even better as their recognition rates become very similar to those obtained by the sighted users. Finally, it was observed that the better recognized haptic icons are those assigned to the most employed applications by the user.
Learning – Human learning – Visual impairments – Vision – Vibration – Cell phones – Tactile sensation – Vibration engineering
Research on haptics for smartphones has demonstrated that the use of vibrotactile stimuli is an excellent way to transmit complex information [1]. Recently, high-resolution haptic feedback systems for mobile phones have become prevalent, allowing the transmission of rich information through the haptic channel. For example, the TouchSense technology created by Immersion has been recently deployed in LG smartphones [2]. Similarly, Apple devices such as the iPhone 6 s and newer iPhone models as well as the Apple Watch and MacBooks from 2015 implement the Taptic Engine as part of their Force Touch technology [3].
Initially, most vibrotactile stimuli used in typical mobile phones transmitted very simple information, such as alerts. However, modern operating systems such as iOS allow the assignment of vibration sequences (predefined or user defined) to contacts in the agenda or to native services. A similar functionality can be achieved in Android-based smartphones using applications such as Good Vibrations [4]. This application allows the configuring of vibration patterns not only for different applications but also for specific contacts.
Adding complex vibrotactile stimuli to smartphones is a key advantage for users with some type of sensory deficiency, such as users that are deaf or visually impaired (VI). This technology allows improved communication using the sense of touch to compensate or even substitute for deficiencies in other senses [5][6]. In addition, the vibrotactile stimuli also increase privacy, as the vibrations are only perceived by the user. These stimuli also improve the user’s experience as they employ an additional channel for communication apart from the standard ones (audio and visual channels), and in some cases, such as in loud environments (e.g., a concert), they could substitute for the audio channel [7].
To communicate complex information, the vibrotactile stimuli need to be associated with specific meanings. These meaningful tactile stimuli are called tactons (tactile icons) or haptic icons. Tactons are defined as structured abstract messages that can be used to communicate messages non-visually [8]. Haptic icons use a metaphorical approach to establish an association between the vibrations and the concept [9].
In a previous study [10], an application (ConTactos) was presented that associated the haptic icons and the vibration pattern using a metaphor. For instance, the vibration associated with the contact called ‘partner’ was a heartbeat, and the vibration of the contact called ‘neighbours’ was a knocking door. This kind of metaphor is not used here because the current work allows associating the haptic icons with the alerts freely and not in a predetermined way.
Vibration signals are mainly employed as a way to transmit alert information as they have the capability to attract the user’s attention even in loud environments. Consequently, they are ideal for notifications such as for incoming calls or meetings. These notifications are usually composed of a simple vibration to attract the user’s attention but without any other type of related information.
The human cognitive system imposes potential limitations for the use of haptic icons [11,12], and hence, their design should allow the information-processing limitations to be clearly perceived and their meaning easily learned [13][14][15].
Currently, some tools and development kits allow for the easy development of haptic icons. Android Haptics [16] is a section of the Material Design Guidelines, Components and Tools that support the best practices of user interface design for the Android operating system. Texas Instruments has developed the TI Haptics technology [17]. It is composed of a set of hardware actuators and drivers that are controlled and configured by the Haptic Control Console software.
Learnability is one of the key aspects when designing and implementing haptic icons. Consequently, understanding the process of learning the meaning of a set of haptic icons has been an important issue in haptic icon research [18]. From the beginning of the use of computer graphical interfaces, there has been a set of designed icons that are now easily recognized and identified by users. However, icons are not inherently meaningful, and users must learn the link between the stimulus and its meaning. For instance, the save icon is usually represented with a floppy disk, although young people have probably never used one. However, they learn that this icon is employed to save even though there is not a clear correspondence for them about the meaning of the image. The link between the stimulus and its meaning can range from representational to abstract. Some icons are purely representational, linking objects and their meanings. For instance, a tweet sound could indicate the reception of a message from the Twitter application. Other icons are abstract without intrinsic meanings but are associated with arbitrary relationships that have to be learned, such as Windows error sounds [19].
To learn the association between a haptic icon and its meaning, in the work of Enriquez et al [20], a three-stage approximation was proposed. The first stage is a self-guided learning process in which users learn the association between the haptic stimuli and its meaning. In this stage, the haptic stimuli are presented along with a representation of their meaning. This representation can be a textual, audio, and/or a graphical description. The second stage is based on reinforcement; in other words, the user must recognize the associated meaning of a haptic stimulus and receive feedback on whether the answer is correct. The last stage is the same as the second stage but without feedback.
In the study of Adam Swerdfeger [21], a unimodal learning process was employed in which a textual description was displayed on the screen next to a button. This button triggers the associated haptic stimulus when pressed. However, multimodal learning processes are advantageous, as stated in several learning theories such as the dual coding theory [22] or the multimedia learning theory [23].
In recent years, game dynamics have been employed to obtain better results in the learning process [24]. This technique is named gamification and can also be applied to help visually impaired people learn to use a smartphone. The usage of games to improve learning capacity is a line of research currently in development [25]. In the work of [26], a gamified mobile application, oriented toward elderly or mildly visually impaired people, was implemented. In [27], a videogame named Audio Haptic Maze was designed, implemented and evaluated. It was oriented toward blind people and used sounds as well as haptic interfaces to guide the user to the exit of a maze. An application that assists visually impaired children in learning to type words was presented in [28], while the work of [29] developed an application for reinforcing the binary calculation skills of engineering students.
For non-visually impaired people, haptic icons can support (or substitute if desired) the visual sensation; meanwhile, for visually impaired people, haptic icons can provide them with an alternative form of sensory information. Indeed, as the number of VI people who use mobile technologies increases [30] several applications that use haptic icons to communicate information have been developed, including an assistive technology that uses the haptic channel to give feedback to the user being guided [31].
It has been noticed that visually impaired people outperform sighted people in terms of recognition when using the haptic information due to cognitive-processing differences [32]. Some authors have provided empirical evidence, from a neurophysiological perspective, stating that people without the ability to see can compensate for this by enhancing their other senses, that is, sensory compensation [33]. For instance, significant sensory differences have been identified between visually impaired and sighted people in terms of their tactile sensory threshold [34]. In addition, some studies [12] [35] [36] have demonstrated that visually impaired people have better memorization skills and show better recognition performance at haptic tasks than sighted people.
In this study, a mobile application has been designed to assist users in learning the association between a set of vibrotactile stimuli and the alarms and notifications of a set of commonly used mobile applications. This application uses gamification and a reinforcement stage to improve the learning process.
Based on the literature reviewed, our hypotheses were as follows:
A reinforcement stage could improve the recognition rate as well as the subjective perception and distinction of the haptic icons.
The use of haptic icons can benefit visually impaired users.
The use of haptic icons associated with smartphone application alerts improves the user experience.
Both improvements also apply to visually impaired users.
Forty-six participants of ages between eighteen and sixty performed the experiments. Eighteen of the participants were visually impaired so that they could not use graphical user interfaces, and twenty-eight of them were fully sighted non-visually impaired users. All of them were smartphone users, those with vision problems used applications such as the Android Accessibility Suite as a way to interact with their smartphones. This suite is usually preinstalled in Android-based smartphones.
The data obtained were analysed anonymously and the study was reviewed and approved by the Experimentation Ethic Committee of the University of Málaga. They are all available from S1 File.
This study followed a 2x2 design, taking the user’s visual capacity (VI vs. non-VI) and the learning process (with and without reinforcement) as independent variables. The dependent variables were the time taken to answer the test, the proportion of recognized haptic icons and the results obtained by a usability questionnaire.
In the reinforcement learning scenario, some participants performed a previous learning stage in which they had to pair haptic icons with the application alerts. They received feedback informing them if their answer was right or wrong. This stage was named learning reinforcement.
Half of the VI subjects performed the reinforcement stage as well as half of the non-VI subjects. Consequently, nine VI and fourteen non-VI subjects executed the reinforcement stage and nine VI and fourteen non-VI subjects did not use reinforcement.
An Android application named HAPP (Haptic APPlication) has been developed to perform this study. Its source code can be accessed online (https://github.com/Equinoxe-fgc/HAPP). HAPP implements a set of sixteen haptic icons each of which is associated with an application alert for the smartphone. The haptic icons have been designed using the Haptic Studio tool [37], and they follow rhythmic patterns with a short duration (2–4 seconds) to easily memorize and distinguish them. Hence, they do not follow any predefined meaning or relationship with the application they represent.
The main parameters to take into consideration for the design of haptic icons are the frequency, amplitude, waveform and duration of the vibrotactile stimulus [38] [39]. However, haptic icons must be designed to be easily recognized, and hence, a more complex combination of those parameters can be employed. Musical techniques are useful for obtaining sets of identifiable haptic icons that are distinguishable from one another. For instance, rhythm, defined as a group of pulses of different durations and amplitude, have been proven to be easily remembered [40]. In addition to rhythm, other musical techniques have been applied to create sets of haptic icons, such as the use of crescendo, defined as a steady increase in force or intensity [41] and melody, defined as a variation in rhythm, frequency and amplitude [21].
Fig 1 shows the vibration pattern for the haptic icons used by the HAPP application.
The Y axis represents the intensity of the signal from 0 to 1, with a value of 0 representing the absence of a vibration and a value of 1 representing the maximum vibration intensity. The X axis represents the time, in milliseconds, that the vibration lasts. Most of the patterns use combinations of full vibration intensity and no vibration. However, the E-Mail, Facebook and SMS applications also employ lower vibration values (between 0,2 and 0,3). The Battery and Camera applications use sforzandos, defined as a high intensity vibration followed by a silence and a crescendo or decrescendo in the intensity of the vibration. The use of sforzandos in the design of haptic icons was studied in [41] where it was concluded that they should not be used because they caused confusion and were not clearly differentiated. Nevertheless, this work did not consider the effect of training, and it is proposed as future work. Finally, Twitter uses the ASDR (Attack-Sustain-Decay-Release) technique, where the vibration magnitude increases from 0 to the highest level (Attack), and it is maintained for a certain amount of time (Sustain). Afterwards, the vibration intensity decreases (Decay) and is maintained again to finally decrease to 0 (Release).
To evaluate the distinctiveness of the designed haptic icons, we performed a multidimensional scaling study. The selected vibration attributes are as follows:
Number of pulses: defined as the number of vibrations between the absences of vibration.
Number of repetitions: The number of times a pattern is repeated.
Duration: The time that a vibration lasts in seconds.
Softness: This parameter indicates whether the vibration has any crescendo or decrescendo (value 3) or not (value 1).
Sharpness: Similar to softness, this parameter indicates whether the vibration has any abrupt change from no vibration to high vibration (value 3) or not (value 1).
Crescendo: The Battery and Camera have the same vibration characteristics except for the use of a crescendo or decrescendo for the intensity of the vibration. Hence, this parameter indicates whether the vibration has a crescendo (value of 3) or not (value of 1). In addition, the Twitter haptic icon has both a crescendo and decrescendo, although the crescendo time is longer than the decrescendo time.
Two-level intensity: A value of 3 indicates that the vibration has two different levels of intensity; otherwise, the intensity is the highest.
Frequency dispersion: This parameter defines the variation in the frequency of the vibration. It is calculated as the quotient between the maximum and minimum periods of the vibration.
We consider the selected parameters to be sufficient to characterize the difference between the vibration patterns. The values obtained to develop the study are shown in Fig 2.
We used the Euclidean distance to perform a multidimensional scaling study and obtained a stress value of 0,01, which indicates a good fit as it is very close to zero. The representation of the multidimensional scaling is shown in Fig 3.
It can be observed that the distance between the haptic icon vibrations is enough to be differentiated. The Update, Calendar, Google+ and Twitter haptic icons as well as those for the pair composed of Battery and Camera are very distinguishable as they are very far from each other. On the one hand, the Update, Calendar and Google+ haptic icons employ a rhythmic pattern that is not used in the rest of the haptic icons. On the other hand, the Battery haptic icon employs a decrescendo, the Camera haptic icon employs a crescendo and the Twitter haptic icon employs both a crescendo and decrescendo.
Apart from the mathematical study of vibration distinguishability, we have also performed a perceptual similarity map study with five subjects to confirm the distinguishability assumption. Each subject had to use the haptic icons in pairs and fill out a perceptual table in which each position represents the distinguishability of a pair of icons. The selected values range from 1 to 5. A value of 1 indicates that the two vibration patterns are almost undistinguishable. In contrast, a value of 5 indicates that the vibration patterns compared are clearly distinguishable.
Fig 4 depicts the obtained perceptual similarity map where each value is the mean of the values selected by the five users. All the obtained perceptual similarity maps are available in S1 File.
Values greater than 4 are shown with a green background, indicating that the vibration pair is easily distinguishable. Values ranging from 3 to 4 are depicted using a yellow background. Values lower than 3 are shown in red, indicating that they could be difficult to distinguish.
The pairs Battery-Camera and Camera-Twitter obtained the lowest values; these are the haptic icons that use a crescendo and/or decrescendo. The Battery haptic icon uses a short vibration pulse and a decrescendo, while the Camera haptic icon uses a short vibration and a crescendo both with the same duration. It seems that people have difficulty in differentiating these vibrations, as was studied in [41], although the multidimensional scaling representation showed that the vibrations are distinguishable. Nevertheless, these icons will not be discarded as we are going to study the influence of the reinforcement learning process as suggested by [42]. Surprisingly, the Camera-Twitter haptic icon pair is shown to be very far apart in the multidimensional scaling representation, and apart from the Twitter haptic icon, it is the only one that has an ADSR vibration pattern; these icons present the lowest perception value. Despite the apparent recognition problems obtained, we decided to maintain the initial haptic icon design as the mathematical distinguishability test obtained very different results.
The HAPP application implements visual icons, haptic icons, and audio described help to be usable by visually impaired people. The usage of screen read tools such as TalkBack was rejected as they interfere with the application usability. Hence, an interface that emulates the help provided by TalkBack has been developed for visually impaired people.
The application also implements an event log to register the time of all the touches performed by the user while the application is in use. The event log records the instant when an event is triggered (touch an icon) as well as the icon touched. This allows for the deduction of the usage speed and the success ratio when the icons are paired.
Prior to the sessions, the participants were instructed to memorize the association between a set of sixteen vibrotactile stimuli and their associated application. The experiment should determine if the usage of complex vibrations obtains a good success ratio relative to the capacity to learn the association between the haptic icons and the application alerts and if the usage of a reinforcement stage improves this success ratio. In addition, the experiment should also determine whether both improvements apply to visually impaired people, and if so, whether they could be employed to improve the smartphone accessibility for people with visual disabilities.
Following the learning process proposed by [20], the application usage was divided into three stages: two learning stages and one stage where the subjects filled in a form designed to determine how easily they remembered the associations between the haptic icons and the application alerts.
The usage stages are as follows:
In the first learning stage, sixteen haptic icons are shown to the subject for their memorization. In this stage, the subject explores the smartphone screen, and a recorded description associated with each application is played when the subject touches the haptic icons. When a haptic icon is pressed, the smartphone plays the vibration associated with the haptic icon of this application. Fig 5 shows the screen for this learning stage of the application.
In the reinforcement learning stage, the subject plays a game with the aim of improving the recognition of the haptic icons. The objective of the game is to pair the applications with their corresponding haptic icon as in the pair card game. In this stage, two screens with a grid of the sixteen icons were shown to the user. The eight icons located at the top of the screen vibrate when they are pressed according to their related haptic icon. The eight icons located at the bottom of the screen are the application icons, and they are initially hidden by a question mark (?) symbol. When a pair of icons are pressed (one of the top group and one of the bottom group), a voice says the name of the application and whether the pairing is correct. If the pairing is correct, the application icon will be shown face up as in the card game, and a voice saying the name of the application will be played when it is pressed again. If the pairing is wrong, the icons are hidden again. To be usable by people with a visual disability, different feedback sounds (beeps) are played depending on the zone (top or bottom zone), where the screen is pressed (Fig 6).
The test stage has the objective of checking whether the subjects are able to remember the association between the haptic icons and the applications. This stage shows two screens with a grid of sixteen icons, and the subject has to pair the haptic icons with their applications as fast as possible. As in the reinforcement stage, the screen is divided into two sections. The section located at the top of the screen contains eight haptic icons that play the vibrations associated with the application it represents as they are pressed. Similarly, the icons located at the bottom section provide the sound information about the associated application when they are pressed. When two icons are pressed (one from the top section and the other from the bottom section), the icon associated with the application is shown at the top section, and the name of the application is played as a voice message as well as whether the pairing is correct. If the pairing is correct, the haptic icon at the top section will remain visible, and the icon located at the bottom section will be hidden. On the other hand, if the pairing is wrong, the haptic icon will be replaced by a cross icon, and the icon located at the bottom section will be shown again. As in the reinforcement learning stage, different feedback sounds are played (beeps) depending on the zone of the screen that is pressed (top or bottom zone) (Fig 7).
The HAPP application has two modes of operation that enable an evaluation of the capacity of the subjects to memorize the association between the haptic icons and the applications as a function of the learning process (with and without a reinforcement learning stage).
After the test was completed, all the subjects were required to fill out a usability form composed of six questions. Three of the questions were an adaptation of the System Usability Scale (SUS) and Computer System Usability Questionnaire (CSUQ) [43] usability forms. The other three questions were focused on learning-related aspects: perception, differentiation and recognition [40]. A Likert scale from 1 to 7 (from totally disagree to totally agree) was employed for answering the questions on the evaluation form. Table 1 shows the evaluation questions.
In this section, we present the main results of the experiment focusing on the statistically significant results (p < 0.05).
The mean time taken to complete the test stage was 4 : 14 minutes (with a minimum time of 1 : 48 and a maximum time of 10 : 43). The analysis of variance (ANOVA) results confirmed the differences in execution time between the VI and non-VI subjects (F(1,42) = 8.01, p<0.01). These differences are due to the time needed to explore the screen to locate the icons to press.
The two-way ANOVA study of the percentage of recognized haptic icons during the test stage showed significant differences between subjects with different visual capacities (F(1,42) = 13.33, p<0.05) and learning processes (F(1,42) = 25.00, p<0,001). As shown in Fig 8, the recognition rate improves when the reinforcement stage is applied, especially for the VI subjects. The non-VI subjects present a better recognition rate when the reinforcement stage is not applied compared to the VI subjects, although this difference practically disappears when the reinforcement learning is used (F(1,21) = 1.23, p>0.05).
There are significant differences between the recognition rate of the Camera and LinkedIn haptic icons (t45 = 4.29, p<0.001) as well as between the recognition rate of the Twitter and Email haptic icons (t45 = 3.31, p<0.001). Fig 9 represents the recognition rate of VI and Non-VI subjects without taking into consideration if they used reinforcement or not. Meanwhile, Fig 10 depicts the recognition rate as a function of the reinforcement without taking into consideration if users were VI or not.
Although the obtained similarity map showed that the Twitter, Camera and Calendar haptic icons could be difficult to distinguish, the results show very different behaviour. These three haptic icons obtained recognition rates close to or greater than 0,8, and were the most recognizable icons in some cases, especially when the reinforcement learning is applied. Consequently, the supposition made by [41] that training users could avoid the misrecognition of sforzandos in haptic icons is confirmed.
Fig 11 shows the recognition rate for each application for the non-VI subjects and compares the results with and without reinforcement learning. Most of the haptic icons have improved recognition rates with reinforcement learning that is maintained, except for the Facebook and WhatsApp applications.
Fig 12 represents the recognition rate for each application for the VI subjects with and without reinforcement learning. The improvement obtained by the VI subjects when using the reinforcement stage is very high for almost every application except for Facebook and LinkedIn where the recognition rate decreases.
If we compare the recognition rate of the VI and non-VI subjects when no reinforcement is applied, Fig 13 is obtained.
The non-VI subjects obtain a better recognition rate than VI for all applications except for LinkedIn. In some cases, such as Camera, WiFi or Update, the difference is very high. However, once the reinforcement learning is applied, this difference is lower or sometimes the VI subjects obtain a better recognition rate, as shown in Fig 14. The VI subjects who did not use the reinforcement stage obtained the lowest recognition rates (between 11% and 56%). In contrast, the non-VI subjects who did use the reinforcement stage obtained the highest rates (between 40% and 100%).
Both the VI and non-VI subjects improve their recognition rates. Even the applications, such as Twitter, Phone, Line, Update or Camera, obtain rates close to 1.
We performed non-parametric Mann-Whitney U tests to compare the replies to the usability form in terms of the use or not of the reinforcement learning stage for the VI and non-VI subjects.
Considering the reinforcement learning condition, the VI users stated that they were able to better recognize the meanings of the vibrations when the reinforcement learning stage was used (Mann-Whitney U = 17, p<0.05). In contrast, the non-VI subjects did not notice a difference in terms of the reinforcement learning stage (all the answers have p>0.05). However, there were no clear differences in the application usability between the VI and non-VI groups when they used the reinforcement learning stage (p>0.05). However, the visually impaired subjects better distinguished (Mann-Whitney U = 22, p<0.05) and recognized (Mann-Witney U = 28.5, p>0.05) the haptic icons when using the reinforcement stage, although it did not affect the application usability (Mann-Whitney U = 30, p>0.1). Table 2 shows the usability answers from the HAPP application.
In this study, we performed a group of experiments on the learning of a set of haptic icons to identify the alerts for a set of smartphone applications. The results show that learning the set of icons can be performed in less than ten minutes, although this time changes according to the visual capacity of the participant. In the next sections, we discuss the obtained results.
We designed sixteen haptic icons and performed a study to evaluate their distinguishability using a multidimensional scaling study. The obtained results showed that they could be easily recognized because the two-dimensional representation of the distances between them was high enough. However, the differentiation test performed by the subjects showed different results, where pairs of applications, such as Twitter and Camera, were difficult to distinguish. This behaviour could be caused by the absence of a learning process, as the test was performed by the subjects who had not previously used the HAPP application. In fact, the recognition rates of those haptic icons (Twitter and Camera) were greater than 0.7 and, in some cases, close to 1.
Our hypothesis was that a reinforcement learning stage could improve the recognition rate as well as the subjective perception and distinction of the haptic icons. According to the obtained results, we can affirm that the use of a reinforcement stage is positive for the learning process. This claim is derived from the improvement in the recognition rate as well as the answers of the usability form.
First, it should be noted that the VI and non-VI subjects scored the application usability as high whether or not it was used with the reinforcement learning stage. However, the inclusion of the reinforcement learning stage greatly improved the recognition rate of the blind subjects, making their results as good as the results obtained by the non-VI subjects. Moreover, the inclusion of the reinforcement learning stage also improved the subjective perception of the VI subjects on some particular aspects of application usability. This conclusion is derived from the answers given in the usability form, such as the recognition of the vibration meaning and the distinctiveness of the haptic icons.
There is an interesting effect in this test considering the best and worst recognized icons. The subjects were asked why they had recognized some icons very well but others very badly. Many of them answered that they could easily recognize the icons associated with applications that they used frequently, nevertheless, for the applications that they rarely use or are unknown to them, it was very difficult to identify the associated haptic icon.
In this paper, haptic icons were employed to identify the alerts generated by applications on a smartphone. Users with a visual impairment greatly improved their recognition rate and their subjective perception of haptic icon recognition when a reinforcement learning stage was added to the learning process.
All the subjects that performed the tests agreed that it was useful to associate different vibrations with the application alerts on the smartphone, although some of the subjects stated that they would only associate the vibrations with those applications that they used frequently. This suggestion is very interesting as it indicates that the utility of the haptic icons could be better if they were applied only to the most commonly used applications or if the user was given a choice to establish a relationship between a haptic icon and an alert.
It should be noted that the subjects did not need much time to learn the associations between the alerts and haptic icons. In addition, the reinforcement learning stage improved the recognition rate of all kinds of users, although this improvement was even better for the VI subjects.
As a future work we propose to test the HAPP application using different smartphones models and test if the recognition rate change due to different perceptions of the vibrations as they may change their intensity and accuracy depending on the smartphone model. Another future development would be to allow the subjects to assign the predefined vibration patterns to the applications and measure if the recognition rate improves.
S1 File [zip] Data files for the perceptual map and recognition per user.
1. Haptics White Paper : Improving the Mobile User Experience through Touch [Internet]. 2010 [cited 2019 Jan 23]. https://pdfs.semanticscholar.org/7e1a/fc2a56f22eec3b4790a4cbba8a669054ec64.pdf
2. Immersion and LG Electronics Expand Business Relationship to Include HD TouchSense Technology [Internet]. 2017 [cited 2019 Jan 23]. https://www.businesswire.com/news/home/20170809005178/en/Immersion-LG-Electronics-Expand-Business-Relationship-Include
3. The science behind Force Touch and the Taptic Engine [Internet]. [cited 2019 Jan 23]. https://www.imore.com/science-behind-taptics-and-force-touch
4. Good Vibrations [Internet]. [cited 2019 Feb 22]. https://wbouvy.com/goodvibrations
5. Rantala J, Raisamo R, Lylykangas J, Surakka V, Raisamo J, Salminen K, et al. Methods for Presenting Braille Characters on a Mobile Device with a Touchscreen and Tactile Feedback. IEEE Trans Haptics. 2009;2(1):28–39. doi: 10.1109/TOH.2009.3 27788094
6. Csapó Á, Wersényi G, Nagy H, Stockman T. A survey of assistive technologies and applications for blind users on mobile platforms: a review and foundation for research. J Multimodal User Interfaces. 2015;9(4):275–86.
7. Hoggan E, Crossan A, Brewster S, Kaaresoja T. Audio or tactile feedback: Which modality when? In: Proceedings of the 27th International Conference on Human Factors in Computing Systems. Boston, USA; 2009. p. 2253–6.
8. Brewster S, Brown LM. Tactons: Structured tactile messages for non-visual information display. In: 5th Australasian User Interface Conference. Dunedin, New Zealand; 2004. p. 18–22.
9. Maclean K, Enriquez M. Perceptual design of haptic icons. In: Eurohaptics 2003. Dublin, UK; 2003. p. 351–63.
10. María Galdón P, Ignacio Madrid R, De La Rubia-Cuestas EJ, Diaz-Estrella A, Gonzalez L. Enhancing mobile phones for people with visual impairments through haptic icons: The effect of learning processes. Assist Technol. 2013;25(2):80–7. doi: 10.1080/10400435.2012.715112 23923690
11. Gallace A, Tan HZ, Spence C. The Body Surface as a Communication System: The State of the Art after 50 Years. Presence. 2007;16(6):655–76.
12. Gallace A, Spence C. The cognitive and neural correlates of tactile memory. Psychol Bull. 2009;135(3):380–406. doi: 10.1037/a0015325 19379022
13. Ternes D, MacLean KE. Designing large sets of Haptic icons with rhythm. In: EuroHaptic 2008. Berlin, Germany; 2008. p. 199–208.
14. Azadi M, Jones LA. Evaluating vibrotactile dimensions for the design of tactons. IEEE Trans Haptics. 2014;7(1):14–23. doi: 10.1109/TOH.2013.2296051 24845742
15. Jimenez R, Jimenez AM. Sequential Recognition Rate and Latency of Frequency-Based Tactons. In: 1st International Conference on Human Systems Engineering and Design: Future Trends and Applications. Champagne, France; 2018. p. 16–20.
16. Android Haptics [Internet]. [cited 2019 Feb 5]. https://material.io/design/platform-guidance/android-haptics.html#
17. TI Haptics [Internet]. [cited 2019 Feb 5]. http://www.ti.com/motor-drivers/actuator-drivers/overview.html
18. Maclean KE, Hayward V. Do It Yourself Haptics: Part II [Tutorial]. IEEE Robot Autom Mag. 2008;15(1):104–19.
19. Stephan K, Smith SE, Martin RL, Parker S, McAnally KI. Learning and Retention of Associations Between Auditory Icons and Denotative Referents: Implications for the Design of Auditory Warnings. Hum Factors. 2006;48(2):288–99. doi: 10.1518/001872006777724426 16884050
20. Enriquez M, MacLean K. The role of choice in longitudinal recall of meaningful tactile signals. In: 16th Symposium on Haptics Interfaces for Virtual Environment and Teleoperator Systems 2008. Reno, USA; 2008.
21. Adam Swerdfeger B. A first and second longitudinal study of haptic icon learnability : the impact of rhythm and melody. Thesis: University of British Columbia; 2009.
22. Paivio A. Mental Representations: A Dual Coding Approach. Oxford University Press, editor. 1990.
23. Moreno R, Mayer R. Cognitive Principles of Multimedia Learning: The Role of Modality and Contiguity. J Educ Psychol. 1999 Jun 1;91(2):358–68.
24. Glover I. Play As You Learn: Gamification as a Technique for Motivating Learners. In: EdMedia + Innovate Learning 2013. Victoria, Canada; 2013. p. 1999–2008.
25. Renaud C, Wagoner B. The Gamification of Learning. Princ Leadersh. 2011;12(1):56–9.
26. Korn O. Industrial playgrounds: How gamification helps to enrich work for elderly or impaired persons in production. In: 2012 ACM SIGCHI Symposium on Engineering Interactive Computing Systems. Copenhagen, Denmark; 2012. p. 313–6.
27. Sánchez J, Espinoza M. Audio haptic videogaming for navigation skills in learners who are blind. In: 13th ACM SIGACCESS Conference on Computers and Accessibility 2011. Dundee, UK; 2011. p. 227–8.
28. YanFi, Udjaja Y, Sari AC. A Gamification Interactive Typing for Primary School Visually Impaired Children in Indonesia. Procedia Comput Sci. 2017;116 : 638–44.
29. Salvador-Ullauri L, Jaramillo-Alcázar A, Luján-Mora S. A Serious Game Accessible to People with Visual Impairments. In: 9th International Conference on Education Technology and Computers (ICETC 2017). Barcelona, Spain; 2017. p. 84–8.
30. Griffin-Shirley N., Banda D. R., Ajuwon P. M., Cheon J., Lee J., Park H. R., & Lyngdoh SN. A Survey on the Use of Mobile Applications for People Who Are Visually Impaired. J Vis Impair Blind. 2017;111(4):307–23.
31. Weiss M, Luck M, Girgis R, Pal C, Cohen J. A Survey of Mobile Computing for the Visually Impaired. Cornell University arXiv.org arXiv:1811.10120; 2018.
32. Touch, Representation, and Blindness [Internet]. Debates in Psychology. Oxford: Oxford University Press; 2000. 236 p. http://www.oxfordscholarship.com/10.1093/acprof:oso/9780198503873.001.0001/acprof-9780198503873
33. Cohen LG, Celnik P, Pascual-Leone A, Corwell B, Faiz L, Dambrosia J, et al. Functional relevance of cross-modal plasticity in blind humans. Nature. 1997 Sep 11;389(6647):180–3. doi: 10.1038/38278 9296495
34. Lai H-H, Chen Y-C. A study on the blind’s sensory ability. Int J Ind Ergon. 2006;36(6):565–70.
35. D’Angiulli A, Waraich P. Enhanced tactile encoding and memory recognition in congenital blindness. Int J Rehabil Res. 2002 Jul 1;25(2):143–5. doi: 10.1097/00004356-200206000-00008 12021601
36. Ballesteros S, Bardisa D, Millar S, Reales J. The Haptic Test Battery: a new instrument to test tactual abilities in blind and visually impaired and sighted children. Br J Vis Impair. 2005 Jan 1;23(1):11–24.
37. Immersion Haptic Studio [Internet]. [cited 2019 Apr 4]. https://www.immersion.com/haptic-experience/
38. Jones LA, Kunkel J, Piateski E. Vibrotactile Pattern Recognition on the Arm and Back. Perception. 2009;38(1):52–68. doi: 10.1068/p5914 19323136
39. Ferguson J, Williamson J, Brewster S. Evaluating Mapping Designs for Conveying Data Through Tactons. In: 10th Nordic Conference on Human-Computer Interaction. New York, USA; 2018. p. 215–23.
40. Swerdfeger B, Fernquist J. Exploring melodic variance in rhythmic haptic stimulus design. In: 35th Conference Graphics Interface 2009. Kelowna, Canada; 2009. p. 133–40.
41. M. Brown L, Brewster S, Purchase H. Tactile crescendos and sforzandos: applying musical techniques to tactile icon design. In: Extended Abstracts Proceedings of the 2006 Conference on Human Factors in Computing Systems. Montréal, Canada; 2006. p. 610–5.
42. MacLean KE. Haptic Interaction Design for Everyday Interfaces. Rev Hum Factors Ergon. 2008;4(1):149–94.
43. Tullis T, Albert W. Measuring the user experience: collecting, analyzing, and presenting usability metrics. Morgan Kaufmann Publisher; 2013.
Zvyšte si kvalifikaci online z pohodlí domova
Nemáte účet? Registrujte se
Zadejte e-mailovou adresu, se kterou jste vytvářel(a) účet, budou Vám na ni zaslány informace k nastavení nového hesla.
