Xinhua, the Chinese state news agency, has released its latest artificial intelligence (AI) 3D news anchor. The AI anchor joins a list of growing virtual presenters that are being developed by the agency.
The AI news anchor is named Xin Xiaowei, and it is modeled after Zhao Wanwei, who is one of the agency’s human news presenters.
According to the search engine Sogou, who co-developed the technology, the AI anchor utilizes “multi-modal recognition and synthesis, facial recognition and animation and transfer learning.”
Here comes Xin Xiaowei, the world's first 3D #AINewsAnchor.
Jointly developed by Sogou and Xinhua News Agency, she will report for Xinhua News Agency on the #TwoSessions, creating a new and dynamic viewing experience. pic.twitter.com/5Tok2Mm3Pl
— Sogou Inc. (@Sogou_Inc) May 21, 2020
The video released by Sogou shows Xin Xiaowei speaking on set about how the anchor can “intelligently imitate human voices, facial expressions, lip movements and mannerisms.”
Previous Virtual Presenters
Xin Xiaowei is not the only virtual presenter that has been developed by Xinhua and the Beijing-based Sogou. It joins a growing list that includes their 2018 digital anchor Qiu Hao and 2019 Russian-speaking version.
In 2018, the pair debuted two different AI news anchors, identical to each other in appearance, at the World Internet Conference. The two versions’ biggest difference was language, with one speaking English and the other Mandarin.
Both of the 2018 models were based on Zhang Zhao, who was another human-anchor like Zhao Wanwei.
In order to develop these first models, hours of video footage was used to replicate the movements, expressions, and other features of real-life anchors.
According to a report released by Xinhua in 2018, “AI anchors have officially become members of Xinhua‘s reporting team. Together with other anchors, they will bring you authoritative, timely and accurate news information in Chinese and English.”
The 2018 AI news anchors were used on various distribution channels including WeChat, the TV webpage, Weibo, and Xinhua’s English and Chinese Apps.
The Russian-speaking anchor was released at the St. Petersburg International Economic Forum 2019. It was developed through a different partnership than the other two versions, with Xinhua working with Russia’s leading news agency, ITAR-TASS.
The announcement came as the two nations celebrated their 70th year of diplomatic relations.
ITAT-TASS is one of the largest news organizations in the world, consisting of a network of businesses, media organizations, diplomatic missions, and financial and research institutions. They have over 1,500 reporters present in more than 63 countries.
“We are very excited to launch the world’s first Russian-speaking AI News Anchor,” said Xiaochuan Wang, CEO of Sogou back in 2018. “The development of the Russian-speaking AI News Anchor allows us to share the benefits of Sogou’s leading AI technologies with more diverse audiences around the world. As one of the world’s largest news organizations, ITAR-TASS is an ideal partner for Sogou, and we look forward to introducing this new AI News Anchor to Russian-speaking audiences.”
The Spread of AI Personalities
The newest AI anchor coming from Xinhua and Soguo highlights the increasing presence of AI-personalities, especially in the realm of media. The technology is improving at such a rapid rate that it will soon be undetectable when put next to a real-life human presenter.
The use of these AI-anchors could dramatically alter the media landscape, but it is really just a part of the larger takeover of AI in the industry. Whether it is AI writers, news anchors, or some other use for the technology, it is going to become increasingly difficult to differentiate between what is human-based and what is AI-based.
Dr. Lingjia Tang, CTO and Co-Founder, Clinc – Interview Series
Dr. Lingjia Tang, CTO and Co-Founder of Clinc, is a professor of Computer Science at The University of Michigan. Dr. Tang’s research in building large-scale production infrastructure for intelligent applications is widely recognized and respected in the academic community. In addition to working at both Microsoft and Google, Lingjia received her PhD in Computer Science from the University of Virginia. Lingjia has recently received prestigious awards including ISCA Hall of Fame, Facebook Research Awards and Google research Award.
What initially attracted you to AI? When did you first discover that you wanted to launch an AI business?
In the mid-2000s I was performing research around large-scale systems that support various applications and how we can design servers as a software system to run those applications more efficiently. At the time, we were shifting from working with traditional web applications to more machine learning-driven functions. That’s when I started to pay attention to the algorithms associated with AI and gained interest in fundamentally understanding how AI applications work. Soon after, the research team I was working with decided to pivot and basically build our own AI applications as benchmarks to study, which is what led us to publishing our first few research papers and developing our first product, Sirius—an open end-to-end voice and vision personal assistant.
As an open source software, Sirius allowed people to build conversational virtual assistants on their own. At the time, this was a very limited capability for the general public and was really only controlled by the big companies, such as Google and Apple. However, we saw that we were filling a critical gap when we released the software and saw that it had tens of thousands of downloads in the first week! That was the turning point where we knew there was a lot of market demand for this type of software.
Come 2015, we launched Clinc with the mindset that we would be able to provide everyone – every developer, company, person—who wants to be able to build a virtual assistant with the access to expertise, tooling and innovation to be able to do that.
Clinc offers conversational AI solutions without relying on keywords or scripts. Could you go into some details regarding how this is achieved? What are some of the Natural Language Processing (NLP) challenges that had to be overcome?
What really sets Clinc apart from other conversational AI platforms on the market is its underlying AI algorithms that enable its “human in the room” experience, which understands messy and unscripted language. This allows for corrections to backtrack and “heal” mistakes made in human conversation and enables complex conversational flows—conversations that a real human would be able to understand. In contrast to a speech-to-text word matching algorithm, Clinc analyzes dozens of factors from the user’s input including wording, sentiment, intent, tone of voice, time of day, location and relationships, and uses those factors to deliver an answer that represents a composite of knowledge extracted from its trained brain. For example, if I ask my virtual assistant, “how much money did I spend on a burger?” it needs to understand that I am asking about money and spending, that I am asking specifically about a hamburger and that a hamburger is a type of food and should be matched to my recent spending at a restaurant.
Achieving this level of understanding is not easy. In general, we would break down conversational AI into two components: Natural Language Understanding (NLU) and dialog management. So, the challenge that we had to overcome was figuring out how to build a system that can extract key pieces of information accurately and can anticipate what the user is asking.
We are able to do this through sophisticated, contextual, top-down NLU, that is trained to be intuitive in nature to keep up with the natural flow of conversation, understanding slang and context. This is in comparison to competitive solutions that have a top down, rules-based approach to Natural Language Processing (NLP) that does not allow for conversational healing, meaning if the customer makes an error, the competitive solutions make them go back to square one, wasting time and only frustrating the user. We also use crowdsourcing to extract our language data to create a richer, diverse data set that can be immediately used to train AI models.
Could you discuss how deep learning is used with the Clinc AI system?
Clinc is using a hybrid approach to deep learning where we use the traditional old-school model to some degree and leverage deep learning where needed. Specifically, we use deep learning to understand words and languages to determine the dialogue flow. Generally, our entire dialogue is a combination of deep learning and symbolic AI. We don’t use deep learning for language generation yet because, when it comes to our customers which are primarily in the banking industry, there are a lot of regulations that the virtual assistant must follow that dictate what they can and cannot say to their customers. So, there is still a lot of uncertainty around whether or not the deep learning will be able to follow those set language restrictions.
As of right now, I don’t think the conversational AI community is completely ready to fully adopt deep learning whereas the academic community is 100% all in, but I do look forward to seeing what the new models can do.
What’s the process for a company that wishes to customize the AI’s responses to target a specific audience? Could you give some examples of how Clinc is currently being used by clients?
We allow clients to either license a platform they can build on however they like, or take our fully built and trained chatbot, Finie, and customize it and integrate it into their apps or messaging services. Finie can handle matters related to balances, transactions, spending history, locating an ATM, making a transfer and more.
My favorite example of how a client has customized Clinc’s AI to target a specific audience is İşbank. As Turkey’s largest private bank, they turned to us to develop their digital banking assistant, Maxi, back in 2018. To infuse Maxi with a unique personality, İşbank held 14 focus groups to gauge what sort of traits and skills bank customers wanted in a virtual assistant. They also hired a voice actress to recite sentences in Turkish related to banking tasks. İşbank’s conversational banking team came up with these sentences by considering the way real people would phrase their needs. Upon our recommendation, the team paid participants on crowdsourcing marketplaces such as Amazon Mechanical Turk to supply different ways they might express the same questions, such as a request to view their balances (“what is my balance,” “how much money do I have in my account,” “show me the cash in my account”) or pay a bill (“pay my bill,” “bill payments”).
This example really shows how invested İşbank is in offering a digital banking assistant to help their customers better navigate their accounts. With Clinc, İşbank launched Maxi to more than 7.5 million people, in Turkish. Since its launch, İşbank has seen widespread adoption by more than 5.5 million users, with an average of 9.8 interactions per user. In recent months, as COVID-19 cases increased in Turkey, İşbank swiftly trained Maxi to be responsive to COVID-19-related queries. Since March 2020, Maxi has answered more than 1.2 million customer queries related to COVID-19, a more than 62% increase in usage.
What would you tell women who are interested in learning more about AI but are reluctant to get involved due to it being a male dominated field?
Off the bat, I don’t think there is any reason why AI is considered a male-dominated field. I think there are a lot of women pioneers in AI that are doing really well and are making an impact. I think AI coupled with social policy is a unique area that has the potential to have a lot of impact on people’s everyday lives. This is where I do think more diverse insights across the board would really benefit us, especially since there are a lot of conversations around AI bias involving race and gender. I believe that having a scoped community of AI developers will continue to have a disproportionate impact on society and policy.
For the women out there who are interested in joining the AI field, I highly recommend it especially if you are interested in making an impact. AI has had so much growth and innovation over the years and it really is an exciting time to be a part of it.
Is there anything else that you would like to share about Clinc?
Clinc is making huge strides right now. Personally, I have just stepped into a new role as CTO of Clinc and I am really excited to focus on how we can further work with developers and data scientists to grow the reach of our technology. As I look toward the future, I see the demand for AI-powered applications shifting to enable people who don’t have years of data science experience and machine learning background to be able to use it too. For example, you don’t have to have a graphic design degree to be able to use Photoshop. I think AI is heading in that direction where developers with no AI or machine learning training will be able to achieve results and produce high quality applications. Overall, we want to reiterate that we are not only devoted to the end-user but also to the developers, no matter what level, who show interest in our solution.
Thank you for the great interview, I look forward to followin your progress. Anyone who wishes to learn more should visit Clinc.
Intel AI Powered Virtual Assistant Mobilized to Assist Reopening of Military Museum
A Canadian museum is safely reopening from its pandemic closure with the help of a virtual
The Ontario Regiment Museum houses North America’s largest collection of operational military vehicles, many dating back to the 1940s. The collection allows the public to experience a piece of history, both at the museum and through the historical films in which the vehicles often appear.
At the start of the pandemic in early 2020, CloudConstable began working with the museum to design Master Corporal Lana as an AI virtual assistant who would greet
Before Lana’s deployment, COVID-19 closed the museum to the public. But with over 120 military vehicles that need constant servicing and driving, the museum needed its volunteers to continue their essential maintenance and operations work at the site.
“The Ontario Regiment Museum is one of the few museums in the world with such a large and diverse collection of operating military vehicles, which help people experience history in a very real way. Regular maintenance is crucial, even during the worst of the pandemic, which is why we turned to CloudConstable and Intel to help build an autonomous solution,” said Jeremy Blowers, executive director of the Ontario Regiment Museum.
CloudConstable relied on the Intel RealSense team’s insight that Lana’s existing and unique capabilities — already built on the Intel RealSense Depth Camera and using the Intel® Distribution of OpenVINO™ toolkit for accelerated machine vision inferences — could be extended for a more comprehensive and safer COVID-19 screening solution. Adding an Intel® NUC 9 Pro with Intel Active Management Technology, as part of the Intel vPro® platform, the team reworked Lana to take temperatures via thermal scans and ask a series of questions to assess COVID-19 risk and exposure. Since June, Lana has provided an enhanced, fully automated and touchless screening process so volunteers can continue to do their important work with the vehicles.
“Intel RealSense technology is used to develop products that enrich people’s lives by enabling machines and devices to perceive the world in 3D. CloudConstable leverages Intel’s technology to help create a state-of-the-art natural voice and vision interface with touchless, self-service COVID-19 screening,” said Joel Hagberg, head of product management and marketing, Intel’s RealSense group.
With the Ontario Regiment Museum now preparing to reopen to the public, CloudConstable, along with Intel, is now working to bring the new COVID-19 protection capabilities into the original concept for Lana as a greeter for visitors. Lana will greet visitors, provide contactless check-in, scan temperatures and ensure the museum adheres to visitor limits and other COVID-19 health protection protocols. Eventually, she’ll even thank them for coming and help visitors keep in touch with all the latest activities at the museum.
Dor Skuler, the CEO and Co-Founder of Intuition Robotics – Interview Series
Dor Skuler is the co-founder and CEO of Intuition Robotics, a company redefining the relationship between humans and machines. They build digital companions including ElliQ – the sidekick for happier aging which improves the lives of older adults.
Intuition Robotics is your fifth venture. What inspired you to launch this company?
Throughout my career, I’ve enjoyed finding brand new challenges that are in need of the latest technology innovations. As technology around us became more sophisticated, we believed that there was a need to redefine the relationship between humans and machines, through digital companion agents. We decided to start with helping older adults stay active and engaged with a social companion. We felt this was an important space to start with, as we could create a solution to help older adults avoid loneliness and social isolation. We’re doing this by focusing on celebrating aging and the joys of this specific period in life, rather than focusing on disabilities.
Intuition Robotics’ first product is ElliQ a digital assistant for the elderly. How does ElliQ help older adults fight loneliness, dementia, etc?
90% of older adults prefer to age at home, and we’re seeing a positive trend of “aging in place” at home and within their own communities, as opposed to moving to a senior care facility. We’re also seeing a strong willingness to adopt non-medical approaches to improve quality of life for older adults, including technologies that allow them to thrive and continue living independently, rather than offerings that only treat issues.
Many home assistants on the market today are reactive and command-based; they only respond to questions and do tasks when prompted. This does little to create a relationship and combat loneliness as you feel like you’re just talking to a machine. ElliQ is different in that she intuitively learns users’ interests and proactively makes suggestions. Instead of waiting for someone to ask her to play music, for example, ElliQ will suggest digital content like TED talks, trivia games, or music. She’ll learn her user’s routines and preferences and will prompt them to engage in an activity after ElliQ notices inactivity. ElliQ creates an emotional bond and helps users feel like they aren’t alone.
You’ve stated that pro-active AI initiated Interactions is very important. In one product demo one of the interesting functions is ElliQ will randomly introduce a piece of interesting information. Is this simply a way of connecting with the user? What are some of the other advantages of doing this?
Proactivity helps to create a bi-directional relationship with the user. Not only is the user prompting the device, but since the device is a goal-based AI agent wanting to motivate the user to be more connected and engaged in the world around him, she’ll proactively initiate interactions that will promote the agent’s goals. Proactivity also helps the user in relating better to the device and feeling as if this is a lifelike entity and not a piece of hardware.
Being pro-active is important, but one of the challenges of a digital assistant is not to annoy a user, how do you tackle this challenge?
We have been designing our digital companions to encompass a “do not disturb the user” goal. This goal is part of our decision making algorithm based on which the agent makes a decision what to proactively initiate. This goal competes with the agent’s other goals such as keeping the user entertained or connected to family members. Based on reinforcement learning, one of these goals “wins”.
Can you discuss designing personality or character in order to enable the human to bond with the machine?
A distinct, character-based personality makes an AI agent more fun, intriguing, and approachable, so the user feels much more comfortable opening up and engaging in a two-way exchange of information. The agent’s personality also provides the unique opportunity to reflect and personify the brand and product that it’s embedded into – we like to think of it as a character in a movie or play. The agent is like an actor that was selected to play a specific role in a movie, serving its unique purpose in its environment (or “scene”) accordingly.
As such, the agent for a car would have a completely different personality and way of communicating than an agent designed for work or home. Nevertheless, its personality should be as distinct and recognizable as possible. We like to describe ElliQ’s personality as a combination of a Labrador and Sam from Lord of the Rings – highly knowledgeable, yet loyal and playful. Discovering the agent’s personality over time helps the user open up and get to know the agent, and the enticement keeps the user coming back for more.
Sometimes an AI may interrupt a conversation or some other event. Is ElliQ programmed to ask for forgiveness? If yes, how is this achieved without further annoying the end user?
ElliQ’s multi modality allows her to express her mistakes. For example, she can bow down her head to signal that she’s apologetic. Overall in designing an AI agent, it is very important to create fail and repair mechanisms that will allow the agent to sophisticatedly apologize for disturbing or not understanding.
One of the interesting things you stated is that ‘users don’t want to anticipate what she (ElliQ) will do next’. Do you believe that people yearn for the type of unpredictability that is normally associated with humans?
We think that users yearn for many elements of human interaction, including quirks like unpredictability, spontaneity, and fun. ElliQ achieves this with unprompted questions, suggestions, and recommendations for activities in the digital world and the physical world. To increase the feeling of having a machine, ElliQ is designed to not repeat herself but to surprise the users with her interactions. This is all to invoke the feeling of being lifelike and to allow the creation of a real bond.
Users will learn to expect ElliQ to anticipate their needs. Do you believe that some type of resentment towards the AI can begin to brew if this anticipation remains unmet?
Yes, I think users will be disappointed if AI around them doesn’t act on their behalf or expectations are unmet. This is why transparency is important when designing such agents – so the user really understands the boundaries of what is possible.
You also stated that users do not see ElliQ as something that is alive, instead they see it as an in-between, something not alive or a machine, but something closer to a presence or a companion. What does this tell us about the human condition, and how should people who design AI systems take this into consideration?
This tells us that as humans, we need interaction and to build relationships in order to feel connected. ElliQ won’t replace other humans, but she can help evoke similar feelings of companionship and help users not feel so lonely or like they’re just talking to a box. She’s much more than an assistant or a machine; she’s a companion with a personality. She’s an emotive entity that users feel as if she lifelike but they truly comprehend that she is actually a device.
Intuition Robotics also has a second product which is an in-car digital companion. Could you give us some details about this product?
In 2019, Toyota Research Institute (TRI) selected Intuition Robotics to collaborate on an in-car AI agent. Through this collaboration, we’re helping TRI create an experience in which the car will engage drivers and passengers in a proactive and personalized way. The experience is powered by our cognitive AI platform, Q. It’s an in-cabin digital companion that creates a unique, personalized experience and aims to accelerate consumer’s trust with autonomy in cars and create a much more engaging and seamless in-car experience.
Thank you for the interview, I really enjoyed learning more about your company and how elliQ can be such a powerful solution for an elderly population that technology often ignores. Readers who wish to learn more should visit Intuition Robotics, or visit ElliQ.
- Andrew Stein, Software Engineer Waymo – Interview Series
- Michael Schrage, Author of Recommendation Engines (The MIT Press) – Interview Series
- Scientists Detect Loneliness Through The Use Of AI And NLP
- Engineers Develop New Machine-Learning Method Capable of Cutting Energy Use
- Artificial Intelligence Enhances Speed of Discoveries For Particle Physics