Big data is the term for a collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications. The challenges include capture, creation, storage search, sharing, transfer, analysis and visualization. At multiple TERABYTES in size, the text and Images of Wisped are a classic example of big data.
As of 2012, limits on the size of data sets that are feasible to process In a reasonable amount of time were on the order of Gigabyte of data.Big data Is difficult to work with using most relational database management systems and desktop statistics and visualization packages, requiring Instead "massively parallel software running on tens, hundreds, or even thousands of servers". What Is considered "big data" varies depending on the capableness of the organization managing the set, and on the capableness of the applications that are traditionally used to process and analyze the data set In Its domain? For some organizations. Facing hundreds of gigabytes of data for the first time may trigger a need to reconsider data management options. For others, it may take tens or hundreds of terabytes before data size becomes a significant consideration.
" Big Data usually includes data sets with sizes beyond the ability of commonly used software tools to capture, curate, manage, and process the data within a tolerable elapsed time.Big data sizes are a constantly moving target, as of 2012 ranging from a few dozen terabytes to many potables of data in a single data set. Garner, and now much of the industry, continue to use this "vs." model for describing big data.
In 2012, Garner updated its definition as follows: "Big data is high volume, high elicits, and/or high variety information assets that require new forms of processing to enable enhanced decision making, insight discovery and process optimization. Additionally, a new V "Veracity" is added by some organizations to describe it. International development: "Big Data" has increased the demand of information management specialists in that Software GAG, Oracle Corporation, MM, Microsoft, SAP, EMCEE, HP and Dell have spent more than $15 billion on software firms only specializing in data management and analytics. In 2010, this Industry on its own was worth more than $100 billion and was growing at almost 0 percent a year: about twice as fast as the software business as a whole .Architecture: In 2004, Google published a paper on a process called Map Reduce that used such architecture.
Map Reduce framework provides a parallel processing model and associated Implementation to process huge amount of data. The framework was Incredibly successful, so others wanted to replicate the algorithm. Therefore, an Implementation of Map Reduce framework was adopted by an Apache open source project Named Hoodoo Advantages&disadvantages: analytics, but big data analytics practitioners as of 2011 did not favor it.
