Optimizing hadoop parameter settings with gene expression programming guided PSO

Created by W.Langdon from gp-bibliography.bib Revision:1.4524

  author =       "Mukhtaj Khan and Zhengwen Huang and Maozhen Li and 
                 Gareth A. Taylor and Mushtaq Khan",
  title =        "Optimizing hadoop parameter settings with gene
                 expression programming guided {PSO}",
  journal =      "Concurrency and Computation: Practice and Experience",
  year =         "2017",
  volume =       "29",
  number =       "3",
  keywords =     "genetic algorithms, genetic programming, gene
                 expression programming, PSO",
  bibdate =      "2017-05-20",
  bibsource =    "DBLP,
  DOI =          "doi:10.1002/cpe.3786",
  abstract =     "Hadoop MapReduce has become a major computing
                 technology in support of big data analytics. The Hadoop
                 framework has over 190 configuration parameters, and
                 some of them can have a significant effect on the
                 performance of a Hadoop job. Manually tuning the
                 optimum or near optimum values of these parameters is a
                 challenging task and also a time consuming process.
                 This paper optimizes the performance of Hadoop by
                 automatically tuning its configuration parameter
                 settings. The proposed work first employs gene
                 expression programming technique to build an objective
                 function based on historical job running records, which
                 represents a correlation among the Hadoop configuration
                 parameters. It then employs particle swarm optimization
                 technique, which makes use of the objective function to
                 search for optimal or near optimal parameter settings.
                 Experimental results show that the proposed work
                 enhances the performance of Hadoop significantly
                 compared with the default settings. Moreover, it
                 outperforms both rule-of-thumb settings and the
                 Starfish model in Hadoop performance optimization",

Genetic Programming entries for Mukhtaj Khan Zhengwen Huang Maozhen Li Gareth A Taylor Mushtaq Khan