 |
MEDAR is an initiative supported by the European Commission following on the work of its predecessor NEMLAR (Network for Euro-Mediterranean Language Resources) |
On April 22nd and 23rd, a consortium of Mediterranean Arabic Language and Speech Technology (MEDAR) actors and players met in Cairo for its 2nd international conference. Beyond the 15 Euro-Med partners, this year’s conference also attracted Columbia University, University of Pennsylvania’s Linguistic Data Consortium (LDC), and Japan’s CJK Dictionary Institute. Companies like IBM, Microsoft, Oracle and Sakhr also figured at the conference.
MEDAR is an initiative supported by the European Commission (EC) and runs from February 2008 through July 2010, following on the work of its predecessor NEMLAR (Network for Euro-Mediterranean Language Resources) which ran from 2003-2005 also under the auspices of the EC. MEDAR is a consortium of Arabic Human Language Technologies (HLT) players. It aims to establish and extend the network of partner centres of best practice in Arabic dedicated to promoting Arabic HLT, surveys present language resource needs, disseminates information on Arabic language technology, establishes development priorities and creates a cooperation roadmap for the region.
Developing Arabic Language Technologies
Work in Arabic HLT took ground some 15 years ago, and, as Dr. Bente Maegaard of the University of Copenhagen observed, was greatly accelerated post 9/11. Arabic is a native to approximately 250 million person around the world. Hence, there are considerable interests and gains to be made by developing and advancing Arabic HLT, not just for applications aimed at intra-Arabic-speaking communities but also for applications that support multilingual interactions. It all starts with research, a deep knowledge of the language, and the sharing of Language Resources (LRs).
LRs are an essential component of language engineering. Basically, what HLT needs first and foremost is raw data, i.e. the language itself – written and spoken words. From the corpora, computational linguists and engineers analyze and systemize the language. This part of the process is considerable as it involves literally breaking down the language and feeding the bits and pieces into machines.
What needs to be done by machines is what humans take for granted with regards to acquisition, learning, understanding, processing and producing language. As Kareem Darwish of Cairo’s Microsoft Innovation Center puts it, HLT is the way that a computer would do things that a human would do naturally concerning natural (human) languages. The current and potential outputs of Arabic HLT, in terms of applications, are staggering.
State-of-the-Art
|
Click here to listen to the interviews with:
1. Dr Khalid Choukri, Co-Chair of MEDAR Conference and Director, Evaluations and Language Resources Distribution Agency (ELDA) France
2. Dr Bente Maegaard, Co-Chair of MEDAR Conference and Director, Centre for Language Technology, University of Copenhagen
3. Kareem Darwish, Researcher, Microsoft Innovation Center
4. Sara Noeman, R&D Engineer, HLT group, IBM
|
What does all the work in HLT mean for end-users? What are the useable products and services available to us? According to LDC’s Executive Director Christopher Cieri, "Mature HLTs are integrated into applications and services like Google that provide better access to information." This was echoed by IBM’s Sara Noeman who also spoke about Google’s Arabic to English automatic translation tool as one that is not yet the most natural of translations, but which provides a good enough readable form.
Microsoft’s Dawish ensured us that Microsoft’s interest in Arabic HLT goes beyond spell and grammar checkers. "The future is in natural interfaces," he says, "People would interact with computers the way they would interact with other people". Microsoft is looking into products and services such as speech-driven navigation through Windows and speech-to-text dictation systems. They are also soon to release a system that allows users unfamiliar with the Arabic keyboard to type transliterated Arabic using Romanised alphabets, which will then be transcribed into Arabic characters.
In other areas, Optical Character Recognition (OCR), the process of converting printed materials into text or word processing files that can be easily edited and stored, has also been tackled. A team of researchers from the University of Sfax in Tunisia looked into Online Arabic Handwriting Recognition, and developed a handwriting recognition system whose results boast of a 95.08% global recognition rate. This is promising for future mainstream and commercial tools.
A collaborative project between LDC and al-Hakawati, the Arab Cultural Trust, is the digitization of Arabic heritage texts. According to LDC’s Cieri, "LDC provides funding and guidance in the selection of texts which al-Hakawati digitizes and makes immediately available via their website. In the meantime, LDC is collecting those same texts into a database that will help scholars understand how the Arabic language has been used over time and in multiple scientific and literary genres." Some of these texts include poetry collections from the Jahiliyyah, texts from ibn Khaldun, al-Jahiz, as well as religious texts like Sahih Bukhari and Muslim.
These examples are but a tiny fraction of the amount of research generated and developed into useful applications. The future looks very bright indeed.
"Imagine", says Dr Khalid Choukri of the European Language Resources Association (ELRA), "that I am speaking Arabic to a Japanese and he listens to me in Japanese, with my voice and intonations." This scenario has already happened in the fictional world of Star Trek. But this previously fictional scenario may not be far from being realized in the real world.
Dr. Choukri means this interpretation to happen via voice recognition, machine translation and speech synthesis, bi-directionally. A team from the Higher Institute for Applied Science and Technology (HIAST) in Damascus, Syria, has been working on text-to-speech processing that involves an emotional audio text-to-speech system that generates speech depending on the input text and the desired emotion type, and an emotional visual model which generates talking heads.
If taken into collaboration with speech-to-speech technology and machine translation, Dr. Choukri’s scenario may very well be a reality in a few years! In fact, Dr. Choukri foresees multilingual innovations, such that machine translations in collaboration with text-to-speech and speech-to-speech technologies may work from and into any number of languages, be they Arabic, English, French, Russian or Mandarin.
Areas of Concern
There are still many areas of concern vis-à-vis Arabic HLT. Firstly, it is an expensive enterprise. As much as possible, MEDAR partners work from present open-source technology and build on it. This helps the Arabic HLT community in terms of sharing knowledge and resources, helping to minimize costs. MEDAR also updates partners on state-of-the-art Arabic HLT so that duplication of work is minimized.
Further, MEDAR encourages and facilitates sharing of knowledge and resources. There was concern raised at the conference that not enough intra-Arab collaboration was taking place and that most of the funding for Arabic HLT has come from Europe and North America and not from Arab countries. Also, according to the most recent MEDAR survey, the majority of players in Arabic HLT are profit-making companies, and universities, with governmental uptake being slow but with promising indicators, as raised by Saudi Arabia’s KACST (King Abdul Aziz City for Science and Technology) and Egypt’s ITIDA (Information Technology Industry Development Agency).
Major international companies like IBM and Microsoft have pumped big money into the field. However, with such economies as Egypt where the IT industry is dominated by SMEs (small and medium enterprises), an over-arching body has to shape the vision and take the lead to help "filter" down the required work. Otherwise development and advancement will largely remain and circulate in the hands of universities and big companies.
The biggest areas of concern are education and literacy, internet penetration and e-infrastructure. Although MEDAR is not working directly in these fields, it is hoped that its work can influence governments to support areas of education and literacy. As Dr Maegaard remarked, a more educated Arabic-speaking population will be able to engineer better products and services that will, in turn, be better used and appreciated. Dr. Choukri would like to see more support for translation projects and publishing of more books. The Arab world combined prints some 12,000 books a year, compared to Turkey, which prints three times more. Internet penetration in the Arab world stands at just under 10%, and MEDAR hopes that through its work in language technologies, a knock-on effect will take place within Arab countries that can improve e-infrastructure and boost internet penetration.
An Accessible, Open, Interactive and Connected World
At the end of the day, the main drive and impetus for Arabic HLT and its equivalents is to make languages and cultures accessible and open to all, and to build applications and machines that can improve, facilitate and help human lives and interactions. With the proliferation and penetration of the internet since the late 80s and early 90s, the world has been increasingly growing bigger, yet closer as information and knowledge becomes readily and easily accessible and available on common platforms such as Google, Facebook and YouTube.
Human interactions, relationships and dialogue are being created, extended and changed fluidly and in ways unimagined even 20 years ago when the first bulky Macintosh Portable was launched and the Berlin Wall came tumbling down. In a world increasingly populated by digital natives and eager digital immigrants, HLT is a key to reduce any gaps arising from linguistic and incompatible platforms.
We have already seen how the internet and its technologies have created a more democratized and level-playing field for participation, creation and innovation across the world, from Tehran to Toronto, from Belarus to Beirut. Imagine what more can happen when Arabic HLT matures and coalesces with multilingual HLTs. The potential for intra-cultural and inter-cultural connections, relationships and dialogue are, frankly, unimaginable.
Even before HLTs mature to become applications, connections have already been made. Arab universities are working hand-in-hand with their European and North American counterparts, multinational companies are making their homes in Arab countries, and slowly but surely, developing countries in the Arab world are making their ICT presence felt on the big stage.
Perhaps it has not been said enough, but technology, and most importantly the internet have arguably been the biggest force for change the modern world has ever seen. We wait with bated breath to the next wave of HLT applications, and will remember the admirable work that MEDAR as a consortium has contributed to the world.
|
