2023.09.22

Free Sign Language Search and Learning App ‘SignTown Handbook’ Released, Uses AI to Recognize SignsSeptember 23 designated by UN as International Day of Sign Languages

Coinciding with the International Day of Sign Languages^*1 on September 23, The Nippon Foundation released the SignTown Handbook sign language learning app on September 10. The app is a simple dictionary that makes it easy to search for and learn signs. SignTown Handbook was released as the second stage of The Nippon Foundation’s Project Shuwa,^*2 which in September 2021 released the SignTown sign language learning game, jointly developed with the Chinese University of Hong Kong and in collaboration with Google and Kwansei Gakuin University.
The SignTown Handbook also ties in with the theme for International Day of Sign Languages 2023, “A World Where Deaf People Everywhere Can Sign Anywhere!” It can be used as a tool to enable many people to learn sign language and for people to have simple conversations in sign language. The project aims to increase the number of people engaged in sign language interpretation and to support the active participation of deaf people in a wide range of activities in the future.

The Nippon Foundation has been supporting the development and promotion of sign language for more than 50 years, focusing on training sign language interpreters and building schools where students can learn sign language, with the aim of having sign language become recognized throughout society at a similar level with English and Japanese. The original SignTown has been used by more than 70,000 people to date, who were introduced to the game through children’s books, school textbooks, and as part of elementary schools’ comprehensive learning programs. In addition to Japanese Sign Language and Hong Kong Sign Language, SignTown Handbook also includes Yangon Sign Language and currently has more than 800 individual entries. A mobile app has been developed in addition to the web-based version, making it free and easy to use on smartphones and tablet devices as well.

SignTown Handbook web version

English version：
https://handbook.sign.town/en/feed?sl=JSL（external site）
Japanese version：
https://handbook.sign.town/ （external site）

SignTown Handbook mobile app

Apple（iPhone・iPad）：https://apps.apple.com/bh/app/signtown-handbook/id1643644150（external site）
Android：https://play.google.com/store/apps/details?id=com.thebitstudio.shuwa&pcampaignid=web_share（external site）

Sign language background

Sign language is a language for mutual understanding among deaf people, and an environment in which sign language can be used is essential to the promotion of deaf people’s participation in society. The Convention on the Rights of Persons with Disabilities, adopted by the United Nations in 2006, clearly states that “‘Language’ includes spoken and signed languages,” and in Japan, the Basic Act for Persons with Disabilities as amended in 2011 recognizes sign language as a language. Nevertheless, understanding of sign language and deaf people is still limited and there are barriers to the use of sign language to participate in various aspects of society. At the same time, the inclusion of sign language interpretation at events like televised press conferences held by public officials when disasters strike around the world has increased interest in sign language, creating an ideal opportunity to promote sign language.

*1. International Day of Sign Languages

The United Nations designated September 23 as International Day of Sign Languages on December 19, 2017, to raise awareness and call attention to the importance of sign languages as languages used by deaf people. September 23 is the date on which the World Federation of the Deaf (WFD) was established in 1951. The WFD is the world’s oldest organization involved with disability issues at the global level, and promotes the use of sign language to enable deaf people to participate in society. Currently, the WFD comprises deaf people’s organizations from roughly 130 countries around the world.

*2. Project Shuwa

Project Shuwa was launched in 2019 to use information and communications technology (ICT) and artificial intelligence (AI) to develop educational materials that make it easy for people to begin to learn sign language, with development jointly caried out by The Nippon Foundation and the Chinese University of Hong Kong, in collaboration with Google and Kwansei Gakuin University. An ordinary camera attached to a personal computer or a smartphone, which can only recognize images in two dimensions, is equipped with special sign language recognition technologies using ICT and AI, making it possible to recognize three-dimensional sign language movements including special actions like body movement, facial expression, leaning forward, and the shape of the mouth.

Project Roles

The Nippon Foundation

Overall project management
Expertise regarding sign language and deaf people
Development funding

The Chinese University of Hong Kong

Joint overall project management
Academic oversight of sign language linguistics
Collection of training data for Hong Kong Sign Language
Expertise regarding deaf people

The Chinese University of Hong Kong website

Kwansei Gakuin University

Collection of training data for Japanese Sign Language
Expertise regarding deaf people

* Kwansei Gakuin University, founded in 1889, is a comprehensive university that provides education based on the principles of Christianity. The Sign Language Research Center (SLRC) was established in April 2015 as the first institute researching sign language at a Japanese university, and has been active since 2016 with support from The Nippon Foundation.

Google

Project concept proposal
Research and development of sign language recognition technologies using artificial intelligence

Contact

Public Relations Team
The Nippon Foundation

Email: cc@ps.nippon-foundation.or.jp