On 3 Nov 2010, Microsoft launched its long-awaited Kinect motion-control system at midnight with a huge dance party in Times Square. Hundreds of professional dancers participated in honor of Kinect’s big game, Dance Central from MTV. The launch is part of more than half billion dollar marketing campaign to promote Kinect and aimed at snaring mass market consumers who play games casually. [1] With pre-orders rapidly sold out and thousands of retailers across the country opened at midnight on launch day, Kinect was a big success. Microsoft has integrated Kinect with its Xbox 360 dashboard. Kinect is available in $150 as a standalone unit or $300 with Xbox-360 console. Just minutes after the launch, Mattrick sold the first Kinect system to a guy named Peter, who had been waiting for days outside the Toys ‘R’ Us store in Times Square, and there were so many like him who waited for days to get a new Kinect system on the very first day. [2] In its first two days of availability Microsoft sold 475,000 Kinect units. [3]In the first 60 days of launch—November 4, 2010 to January 3, 2011—Microsoft Kinect sold 8 million units or an average of 133,333 units per day, which is a Guinness World Record for fastest selling consumer electronics device. Till 10 March 2011, Microsoft sold-through approximately 10 million units of Kinect device.
INDUSTRY’S HISTORY
Video gaming industry is worth over 100 billion dollar worldwide as of 2010, but the modern video game industry had a very humble commencement. [5]The idea for the video game first came in 1951 to Mr. Ralph Baer, who was given the task of developing the television. His idea was to include some interactive game with the television but management didn’t allow him to pursue it further. In the 1950’s and 1960’s the video games were developed in the laboratories and universities in the spare time of students or scientists. During the late 1960’s Ralph Baer again came with the idea of having video games at homes instead of labs and developed an interactive game that could be played on a television and in 1968 he patented the game.
In 1971 the first arcade game was released in the market but it was very difficult to play. In 1972 Magnavox launched the Odyssey, which was designed by Ralph Baer. The Magnavox Odyssey is the first home video game console system. In 1975 Atari released their home version of ‘Pong’ video game which became the hottest selling Christmas present at that time. In 1977 Atari introduces its first cartridge-based home video system called the Video Computer System which later known as the Atari 2600. In 1980 Mattel’s Intelivision debuts and is the first real competitor of the Atari 2600 and it has better graphics than Atari’s 2600. Both the console manufacturers attract third party game developers and Activision becomes the first third-party video game vendor followed by Coleco. In 1983 Nintendo introduces the Famicom in Japan—later known as the Nintendo Entertainment System (NES) in the U.S. Since Atari controls most of the market share in early 1980’s, NES did not plan to market the product in the U.S. Instead Nintendo offers Atari the rights to distribute the product in the U.S. but the deal didn’t go through. In 1985, Nintendo unveiled its American version of the Famicom with a completely redesigned case and a new name, the Nintendo Entertainment System (NES). The NES soon breaks sales records and became the best selling console in the video-game history. After the launch of NES, Nintendo dominated the industry through the rest of the 1980’s with huge hits like Super Mario Bros., Metroid, and The Legend of Zelda. In 1989, Nintendo released a new hand-held gaming system called Game Boy. Game Boy became an instant hit, by selling 32 million units in its initial first three years. Nintendo became the market leader in handheld gaming devices market and longest running company in the industry. Meanwhile in 1986, to compete with the NES, Sega introduced Sega Master System (SMS) and Atari released Atari 7800. In 1989, Sega introduced the Genesis, its first 16-bit home game console, which gave the Nintendo a tough competition. After getting beaten by Sega Genesis, Nintendo released the Super Nintendo Entertainment System (SNES) in 1991 and by the end of the 16-bit era in the mid-90’s, Super Nintendo surpassed Sega Genesis in sales.
The year 1995 was a turning point in the console space with the entry of Sony, which launched the 32-bit Playstation one, the most popular console of the 32-bit era. With a price tag of $299 for the Playstation one, Sony quickly became the market leader. The following year, Nintendo released the Nintendo-64 with titles like Super Mario-64 but it lacked the broad range of games released for the Playstation. In 1999, Sega released 128-bit system the Dreamcast and the Dreamcast was the first console to include a built-in modem and Internet support for on-line gaming. It enjoyed brisk sales in its first season and sold 500,000 consoles in just two weeks. [6]
In 2000, Sony released the Playstation 2, its first 128-bit system with backward compatibility (the ability to play older 32-bit PlayStation games) and also functions as a DVD player. The PS2 became the most popular console of the 128-bit era and also marks the rise in popularity of console games played over internet. In less than a year, more than 10 million PlayStation2 consoles were sold. Looking the success of other players, software giant Microsoft made its first independent foray into the console market with the release of the Xbox in 2001. Microsoft’s superiority in the PC technology allowed the Xbox to perform much better when compared to other 128-bit consoles, but the sales remain far behind that of the PS2. At the same time, Nintendo launched GameCube.
In 2005, Microsoft released Xbox-360 to compete the upcoming generation of consoles, including Sony PlayStation 3 and Nintendo Revolution (Nintendo Wii) launched in 2006. The Wii Remote is a one-handed controller that uses a combination of accelerometers and infrared detection to sense its position in 3D space. The controller connects to the console using Bluetooth and allows users to control the game using physical gestures as well as traditional button presses. Perhaps the most important of these devices is the ‘Nunchuk’ unit, which features an accelerometer and a traditional analog stick with two trigger buttons. Nintendo Wii witnessed the evolution of the controller free gaming at the commercial level.
After years of dominating the motion control gaming arena Nintendo finally had some competition in 2010. The introduction of the Playstation move and later the much hyped launch of the Microsoft Kinect have heated up the competition in motion gaming. Nintendo was the first to unveil a motion controller and the competition is finally catching up with the launch of Microsoft Kinect. Kinect has sold more than 10 million since Nov 4, 2010 till Mar 5, 2011 (4 months) and has become part of Guinness book of world record. It currently holds the title of “Fastest selling consumer electronics device” by Guinness book of world record.
The Nintendo’s latest offering for the Wii was the Wii motion Plus, which allow it to more accurately capture complex motion. This makes experience of playing Wii games a whole lot more realistic. The Controller allows the Nintendo Wii to capture more complex movements which weren’t possible before but Wii motion plus is more like an improvement rather than a significant change.
After that Sony introduced the Playstation Move, which is an accessory exclusive to the Playstation 3 and lets you play motion control games by waving around a control stick that works in conjunction with the Playstation Eye camera. The Playstation Move motion controller has an accessory called the Move navigation controller which consists of a D-pad and an analog stick. The Playstation Move has a 1 to 1 tracking system in 3D space so that the system will track the movements perfectly. The PlayStation Move takes full advantage of the powerful hardware of the PlayStation 3 and gives a full HD experience. [7]
With the Wii Motion Plus and the Playstation Move already in the market, Microsoft also is vying for a piece of the motion gaming pie with the launch of its Kinect for Xbox-360. Kinect created history by becoming the first accessory to bring controller free gaming. All you have to do is get into the Kinect’s range and you are good to go. The Kinect is something revolutionary in the gaming world and would have a wide scope of innovation for future game. [8]
It still has to be seen which platform will be the king of motion gaming. With the Wii having a huge head start, Microsoft and Sony have their work cut out for them.
Figure – Video Game HistoryTECHNOLOGY
Kinect looks like a good artistic visualization, the user will never get bored of looking at how Kinect works “behind-the-scenes”. The Kinect sensor is a horizontal bar connected to a small base with a motorized pivot, and is designed to be positioned lengthwise below the video display. The device features an “RGB camera, depth sensor and multi-array microphone running proprietary software”, which provides full-body 3D motion capture, facial recognition, and voice recognition capabilities. The depth sensor of Kinect consists of an infrared projector coupled with a sensor and allows Kinect to see in 3D under any lighting condition. The sensing range of the depth sensor can be adjusted.
Kinect is based on software technology developed internally by Microsoft and range camera technology by Israeli developer PrimeSense, which interprets 3D scene information from a continuously-projected infrared pattern. The trick used by Kinect is beam forming which is capable of sensing user’s body movements by focusing on certain points on the body, and at the same time comparing it to a skeletal model. [9]
PrimeSense software makes a skeleton of the user which identifies the movements of an object made by the user at a particular time. In June 2008, PrimeSense was interested to sell their technology that would eventually be organized into what we know today as Microsoft Kinect. In Silicon Valley, the first company that PrimeSense CEO, Inon Beracha tried to demonstrate the sensor was none other than Apple. Originally the technology had been developed by engineers in Israeli military.
One shortcomings associated with this gaming tool is that it cannot capture the sensing of the people with dark complexion. As a result, the consumer having a dark complexion have filed lawsuit against Microsoft. After doing testing on several dark skinned people, it is found that it will harm the ability to play games except few small functions that will be inconsistent according to the gaming experts. [10]
In terms of hardware assembling, it includes a RGB camera for facials recognition and video capturing, a depth sensor which is an infrared projector paired with a monochrome CMOS camera, four microphones for voice recognition and a motor in the base for rotation. Pretty colors and dancing skeletons are enamoring, but the hard science behind just how the Kinect on XBOX 360 can recognize human poses in real-time from a single depth-image is evenly beguiling. As a sequence of its development, the research team accumulated a database of approximately 500,000 frames of motion capture data of simulated poses with different people in a different entertainment scenario such as driving, running, kicking, dancing, and navigating menus. [11]
Later on, the research team extrapolated the dataset down to 100000 unique poses to which the system was trained to estimate body parts from. As an indication of just how computational intensive the development process really was, Microsoft has stated that it was like “training 3 (decision) trees to depth 20 from 1 million images takes about a day on a 1000 core cluster”. Microsoft wants its users to play with the Xbox 360 move in front of the screen. To make it effective, they spent hundreds of millions of dollars in developing a device to realize the recognition and the tracking, not only of a complete person but several ones, including 3D information. Here we will discuss how the hardware and the artificial intelligence algorithms work that uses.
To get 3D image, there is a infrared light projector with a camera that captures this kind of light, so the intensity of the infrared light that comes reverberated to the camera, giving to the video console the depth of each pixel, creating a height map in black and white of “320?240” pixels, where the color is more white is usually more near and the color more dark is usually more distant. This method isn’t necessarily the only one to give a machine stereoscopic vision, also can reconstruct a 3D image of the reality using two or more camera, but the chosen method also works with bad illumination conditions.
In addition to that, it also have a colour camera with pixel of 640?480 and CMOS sensor and autofocus to record video of players, makes facial recognition and separate the persons from the background, because the color and textures information is fundamental to accomplish this task. Separate something from background, technically which is called segmentation and before doing so it also have to do an algorithm to align the color image with the height map, knowing the space between the cameras and this is supposed to be calibrated, unless its lenses would provoke different deformation making difficult the task of do the pixel correspondence, while with the calibration the deformations can be deleted before corresponding.
When segmenting an object, Kinect must recognize every part of the object and assign it to a virtual skeleton, where its position will determine the action that will go to realize in the game. In the lower part, there is an engine to calibrate its position with the help of an internal accelerometer, which means that the algorithms use to identify the parts of a person produce errors if the cameras are not seeing just in front of the person. Ended the body recognition, in the following frames will use some algorithm of tracking to know where we are going to move, it will avoid some calculus in the segmentation and recognition of our parts of the body in the next frames.
All artificial intelligence system, it requires a training to learn to recognize us. This consists in leaving the hand in front of the device and making some movements in front of the screen. In the same way, the system extracts the needed patterns to identify the characteristics our hands (skin color, size, etc.) and the characteristics of our body (width, height, etc.).
COMPETITION & COMPETITIVE ADVANTAGE
Presently, there is no direct competition of Kinect. All game consoles have controllers that are either wired or wireless. But if the motion control business segment is concerned, then there are mainly 3 players- Nintendo, Sony and Microsoft.
In 2008, Nintendo came out with a controller called Wii MotionPlus that renders player’s motion in real time. Two years later, Sony came out with PlayStation Move for PS3 and just after it Microsoft created its motion device called Kinect which the user does not need to be held like Nintendo’s motion device or Sony’s device.
Comparison of Nintendo Wii, Microsoft Xbox 360 and Sony PlayStation Move
Feature/NameKinectWiiMove TypeGame ControllerGame ConsoleGame Controller ManufacturerMicrosoftNintendoSony ReleaseNov 4 2010Nov 19 200615th Sept 2010 Required HardwareXBox360 consoleWii RemotePS3, Playstation Eye Media TypeWebcamController and NunchuckMotion sensor and Camera ConnectivityWired to Xbox 360Wi-Fi, Bluetooth and USBBluetooth, USB RecognitionVoice,Face,GesturesMotionVoice,Motion GamesKinect AdventuresWii SportsSports Champions Multiplayer1 Kinect supports 4 playersOne controller for each playerEach player needs a controller CompatibilityXbox360 Slim, older models need power cordBackward compatible with GameCubeAll PS3 models GameplayFast and accurate, very physicalTrendsetter but is slow and inaccurateVery precise, no lag Price$150$60$100Notwithstanding the explosion in the world of motion gaming, Microsoft has an ace in the hole. The researcher termed Kinect something that was “really hard” to develop, and said the power of Microsoft was one of the reasons it was able to come together. The researcher said: “we’ve done with Kinect’s software which is something that’s really hard to pull off. We have a good advantage at Microsoft because we have groups like our Microsoft Research department that not a lot of other companies have, and we’re able to solve some super-complicated technical problems in a short period of time.”
Researcher at Microsoft believes that it’ll be difficult for any other company to replicate Kinect without Microsoft’s know-how. They think it’ll be “super-challenging for anybody else to solve those problems.” [12] This shows that if another company introduces the motion sensing technology, they will not have the first mover advantage. Microsoft will be improving and adding more new technologies to the platform by the time another company comes up with the basic Kinect model. One of the technologies the research team hopes to add in future Kinect games is the digitization of real-world objects first shown off in Kinect’s original demo-reel. Kinect might seem a little fluffy right now, but its software should continue to evolve to stave off competitors.
In order to keep the competitive advantage to them, Microsoft has bought Canesta. Canesta is the rival of Primesense that has co-developed Kinect with Microsoft. This acquisition will enable Microsoft to incorporate 3D sensing and gesture technology of Canesta. This will result in explosion of new devices that will use this technology for businesses and individual users. One of the important reasons is the size of scope that Microsoft offers will help improves this technology and markets it better than smaller companies.
Canesta’s technology has an extra edge over Kinect’s technology since it also incorporates 3D by calculating the distances of the users in real time. Canesta has provided its technology for Sony’s toys but not for Play Station and chances are that it will not anymore, since it’s bought by Microsoft. Other major rivals of Xbox will have to find new suppliers of type of technology used in Kinect or develop their own that will take years.
OTHER USES
Kinetic developed a 3D gesture recognition platform long before Microsoft officially launched Kinect. Despite the head start, the promised consumer gaming and television controllers have yet to appear. There are companies that have venture cash and big league partners whose plan is to attract Kinect hackers into become its developers. A Belgium-based company makes software’s that allow Kinect customers to use applications that do not meet with Microsoft’s approval. Microsoft made some threats about the situation, and eventually backed down. There are other uses like holographic video chat, optical camouflage, merging multiple 3D video streams and more unauthorized Kinect uses arrive daily. Orange Vallee, a Subsidiary of the France Telecom Group created its Keanu, a Kinect-like TV remote control. Netherlands-based Metrological Media Innovations signed on to develop set-top-boxes and offer the technology to television manufacturers.
A new trend is emerging where third party developers are able to change the game’s designs for open source models. Players are able to modify the games according their liking and an example is Counter Strike that is a modification of a popular video game called Half-Life. Due to the success of Counter Strike, it has become a separate game that is very successful and comes up with new versions each year. Consumers have found new ways to modify their Kinect device to use in video games and do other activities like turning on or off their computers with customized gestures. All this has happened when the video game manufacturers released the source code. There will be approximately 600,000 established online game community developers by 2012 that used the open source code to modify games and devices that come with it. Instead of using the Kinect with Xbox, users are turning on and off their lights, playing music from computer and several other activities through their computers. [18]
FUTURE
Undoubtedly Kinect has been a breakout hit for Microsoft Corporation. The sensor in Kinect that uses microphone arrays and a 3D camera to interpret gesture-based movements has out broken new ground when it comes to bringing video games to the common layman. Nevertheless, there are some limitations to the technology. For example, it cannot detect fine movements like the wiggling of object’s fingers. In addition to that, Kinect first has to calibrate before going into action. Of late, gesture-based gaming using 3D cameras is still in its infancy.
The researchers came up regarding the future of this technology with some fascinating insights. One of the common things about 3D cameras is that they use pretty low-resolution cameras right now, and that partly limits the detection of fine movements. Experts say that Kinect cannot recognize each individual’s finger. That’s only good enough to monitor object’s hand and shape of the body. But improvements have been made, and as 3D cameras go higher in resolution, they’ll be able to detect fingers and finer body motions. Being able to detect smaller movements can lead to more game play varieties. For instance, by tracking fingers, a smart developer may create a programme that lets people play piano in the air without a physical keyboard or perhaps players could mold a piece of virtual clay into whatever they want.
The second fact was that 3D cameras are likely going out of consoles and into other devices. One of the first 3D camera-enabled televisions will come from Samsung. Another device that will use it is the Asus Wavi. What does this meanWell, instead of Kinect being the sole mass-market 3D camera, it could be one of millions of devices. According to expert the technology is getting cheaper. “The opportunity to play gesture-enabled games on TVs is three to four years away”. But the big problem facing gesture-enabled interfaces on laptops is that engineers have to figure out how to translate motion that’s a few feet from the camera. According to the same expert engineers are working on the problem, and tablets or laptops that use gesture controls will be appearing in 2012.
REFERENCES http://asia.gamespot.com/news/6282208.html http://venturebeat.com/2010/11/03/microsofts-kinect-motion-control-system-goes-on-sale-with-dance-party-in-times-square/ http://www.crunchgear.com/2010/11/11/early-sales-data-shows-microsoft-sold-425k-kinects-kinect-sports-is-top-selling-game/ http://www.huliq.com/10177/microsoft-kinect-shifts-10-million-units-game-sales-remain-poor http://www.escapistmagazine.com/forums/read/7.228443-Videogame-Industry-Worth-Over-100-Billion-Worldwide http://www.thegameconsole.com/videogames01.htm http://www.pcmag.com/article2/0,2817,2372058,00.asp http://www.gamehoot.com/2010/09/tech-questions/wii-motion-plus-vs-move-vs-kinect-fight.html http://blogs.wsj.com/tech-europe/2010/11/08/key-kinect-technology-devised-in-cambridge-lab/ http://www.geek.com/articles/games/the-technology-behind-the-kinects-voice-recognition-is-ingenious-but-doesnt-work-very-well-2010085/ http://www.computableminds.com/post/kinect/how-works/characteristics/microsoft/xbox-360/3d-body-tracking http://www.escapistmagazine.com/news/view/103294-Microsoft-Kinect-Competitor-Would-Be-Super-Challenging-To-Develop http://www.istartedsomething.com/20110319/microsoft-research-reveals-guts-of-kinect-technology/ Long Zheng 19 Mar 2011 http://www.msnbc.msn.com/id/15734058/ns/business-cnbc_tv/ http://www.techspot.com/news/41019-microsoft-got-kinect-technology-since-apple-is-a-pain-in-the-ass.html http://mashable.com/2011/03/23/kinect-marketing-strategy/ http://www.ithinkdiff.com/microsoft-reveals-xbox-kinect-technology-72123/ http://hothardware.com/News/Kinect-Competitor-To-Bring-Gesture-Control-To-Linux-Windows-And-TV/