Smart grid (SG) is the solution to solve existing problems of energy security from generation to utilization. Examples of such problems are disruptions in the electric grid and disturbances in the transmission. SG is a premium source of Big Data. The data should be processed to reveal hidden patterns and secret correlations to extrapolate the needed values. Such useful information obtained by the so-called data analytics is an essential element for energy management and control decision towards improving energy security, efficiency, and decreasing costs of energy use. For that reason, different techniques have been developed to process Big Data. This paper presents an overview of these techniques and discusses their advantages and challenges. The contribution of this paper is building a recommender system using different techniques to overcome the most obstacles encountering the Big Data processes in SG. The proposed system achieves the goals of the future SG by (i) analyzing data and executing values as accurately as possible, (ii) helping in decision-making to improve the efficiency of the grid, (iii) reducing cost and time, (iv) managing operating parameters, (v) allowing predicting and preventing equipment failures, and (vi) increasing customer satisfaction. Big Data process enables benefits that were never achieved for the SG application.
If the inline PDF is not rendering correctly, you can download the PDF file here.
Acquisto G. Domingo-Ferrer J. Kikiras P. Torra V. de Montjoye Y.A. and Bourka A. (2015). Privacy by Design in Big Data: An Overview of Privacy Enhancing Technologies in the Era of Big Data Analytics. arXiv preprint arXiv:1512.06000 (2015).
Adiba M. Castrejon-Castillo J.-C. Espinosa Oviedo J. A. Vargas-Solar G. and Zechinelli-Martini J. L. Netherlands (2016). Big Data Management Challenges Approaches Tools and their Limitations. Networking for Big Data.
Afrati F.N. Borkar V. Carey M. Polyzotis N. Ullman J. D. (2011). Map-reduce extensions and recursive queries. In: Proceedings of the 14th International Conference on Extending Database Technology Uppsala Sweden 22–24 March 2011 pp. 1–8.
Alexandros L. Jagadish H. V. (2012). Challenges and Opportunities with Big Data. Journal Proceedings of the VLDB Endowment 5(12) pp. 2032–2033.
Béjar Alonso J. (2013). Strategies and Algorithms for Clustering Large Datasets: A Review.
Ben Ayed A. Ben Halima M. and Alimi M. (2014). Survey on clustering methods: towards fuzzy clustering for big data. In: 6th International Conference of Soft Computing and Pattern Recognition (SoCPaR) IEEE Tunis Tunisia 11–14 August 2014 pp. 331–336.
Beyer M. (2011). Gartner Says Solving ‘Big Data’ Challenge Involves More Than Just Managing Volumes of Data. Gartner. Archived from the original on 10 July 2011 [Retrieved 13 July 2011].
Bonomi F. Milito R. Zhu J. and Addepalli S. (2012). Fog computing and its role in the internet of things. In: Proceedings of the First Edition of the MCC Workshop on Mobile Cloud Computing; MCC ’12 New York NY USA ACM 2012 pp. 13–16.
Borkar V. R. Carey M. J. and Li C. (2012a). Big data platforms: what’s next? XRDS Crossroads the ACM Magazine for Students 19(1) pp. 44–49.
Borkar V. Carey M. J. and Li C. (2012b). Inside ‘Big Data Management’: Ogres Onions or Parfaits?
Bredillet P. Lambert E. and Schultz E. (2010). CIM 61850 COSEM standards used in a model driven integration approach to build the smart grid service oriented architecture. In: 2010 First IEEE International Conference on Smart Grid Communications (SmartGridComm) Gaithersburg MD USA 4–6 October 2010 IEEE pp. 467–471.
Cagri Gungor V. Sahin D. Kocak T. Ergut S. Buccella C. Cecati C. and Hancke G. P. (2013). A Survey on Smart Grid Potential Applications and Communication Requirements. IEEE Transactions on Industrial Informatics 9(1) pp. 28–42.
Cattell R. (2011). Scalable SQL and NoSQL data stores. SIGMOD Record 39(4) pp. 12–27.
Chandarana P. and Vijayalakshrni M. (2014). Big data analytics framework. In: Proceedings of the International Conference on Circuits System Communication and Information Technology Applications (CSCITA) Mumbai 4–5 April 2014 IEEE pp. 430–434.
Chandarana P. and Vijayalakslnni M. (2014). Big data analytics framework. In: International Conference on Circuits System Communication and Information Technology Applications Mumbai India 4–5 April 2014 IEEE.
Demchenko Y. Grosso P. De Laat C. and Membrey P. (2013). Addressing big data issues in scientific data infrastructure. In: Collaboration Technologies and Systems (CTS) 2013 International Conference onx San Diego CA USA 20-24 May 2013 IEEE pp. 48–55.
Electric Power Research Institute (EPRI). (2009). Report to NIST on the Smart Grid Interoperability Standards Roadmap. June 2009.
Elluri V. R. and Salim A. (2016). A comparative study of various clustering techniques on big data sets using Apache Mahout. In: 3rd MEC International Conference on Big Data Smart City Muscat Oman 15–16 March 2016 IEEE.
Fadnavis R. A. and Tabhane S. (2015). Big data processing using hadoop. International Journal of Computer Science and Information Technologies 6(I) pp. 443–445.
Fahad A. Alshatri N. Tari Z. ALAmri A. Zomaya A. Y. Khalil I. Sebti F. and Bouras A. (2014). LOOKING BACK of Clustering Algorithms for Big Data: Taxonomy & Empirical Analysis. IEEE Transactions on Emerging Topics in Computing 2(3) pp. 267–279.
Fang X. Misra S. Xue G. and Yang D. (2012). Smart Grid—The New and Improved Power Grid: A Survey. IEEE Communications Surveys & Tutorials 14(4) pp. 944–980.
Ferreira Cordeiro R. L. Traina C. Jr. Traina A. J. M. López J. Kang U. and Faloutsos C. (2011). Clustering very large multi-dimensional datasets with MapReduce. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining ACM San Diego California USA 21–24 August 2011pp. 690–698.
Gungor V. Lu B. and Hancke G. (2010). Opportunities and Challenges of Wireless Sensor Networks in Smart Grid. IEEE Transactions on Industrial Electronics 57(10) pp. 3557–3564.
Hoffmann L. (2013). Looking back at big data. Communications of the ACM 56(4) pp. 21–23.
Jararweh Y. Jarrah M. Alshara Z. Alsaleh M. N. and Al-Ayyoub M. (2014). Cloudexp: A Comprehensive Cloud Computing Experimental Framework. Simulation Modelling Practice and Theory 49 pp. 180–192.
Kersten M. L. Idreos S. Manegold S. and Liarou E. (2011). The Researcher’s Guide to the Data Deluge: ‘Querying a Scientific Database in Just a Few Seconds. Proceedings of the VLDB Endowment 4(12) pp.
Kleiner A. Jordan M. Ameet T. and Purnamrita S. (2012). The big data bootstrap. In: Proceedings of the 29th International Conference on Machine Learning Edinburgh Scotland.
Kusum M. and Rupali M. (2013). A Review on Various Classification Algorithms for An Incremental Spam Filter. International Journal of Application or Innovation in Engineering and Management 2(11) pp. 325–331.
Mandai B. Sahoo R. K. and Sethi S. (2015). Architecture of efficient word processing using hadoop for big data applications. In: International Conference on Man and Machine Interfacing Bhubaneswar India 17–19 December 2015 IEEE.
Markovic D. Zivkovic D. Branovic I. Popovic R. and Cvetkovic D. (2013). Smart Power Grid and Cloud Computing. Renewable and Sustainable Energy Reviews 24 pp. 566–577.
Mohan C. (2013). History repeats itself: sensible and NonsenSQL aspects of the NoSQL hoopla. In: Proceedings of the 16th EDBT International Conference on Extending Database Technology (EDBT’13) Genoa Italy 18–22 March 2013.
Nagpal P. B. and Mann P. A. (2011). Survey of Density Based Clustering Algorithms. International Journal of Computer Science and its Applications 1(1) pp. 313–317.
Oracle Corporation. (2011). Oracle NoSQL Database Compared to MongoDB. White-Paper.
Park K. Nguyen M. C. and Won H. (2015). Web based Collaborative Big Data Analytics on Big Data as a service platform. In: International Conference on Advanced Communication Technology (ICACT) Seoul South Korea 1–3 July 2015.
Reinprecht N. Torres J. and Maia M. (2011). IEC CIM architecture for Smart Grid to achieve interoperability. In: 5th Grid Interop Meeting (Grid Interop) Phoenix USA 2011.
Rohr M. Osterloh A. Gründler M. Luhmann T. Stadler M. and Vogel N. (2011). Using CIM for Smart Grid ICT Integration. IBIS 11(2011) pp. 45–61.
Sadalage P. J. and Fowler M. (2012). NoSQL Distilled: A Brief Guide to the Emerging World of Polyglot. Upper Saddle: Addison Wesley.
Sagiroglu S. and Sinang D. (2013). Big Data: A Review. IEEE 2013.
Saha B. and Srivastava D. (2014). Data Quality: The other face of big data. In: Proceedings of the 2014 IEEE 30th International Conference on Data Engineering (ICDE) Chicago IL USA 31 March–4 April 2014.
Shahrivari S. (2014). Beyond Batch Processing: Towards Real-Time and Streaming Big Data. Computers 3(4) pp. 117–129.
Sherin A. Uma S. Saranya K. and Vani S. (2014). Survey On Big Data Mining Platforms Algorithms and Challenges. Journal of Computer Science & Engineering Technology 5(9) pp. 854–862.
Shirkhorshidi A. S. Aghabozorgi S. Wah T. Y. and Herawan T. (2014). Big data clustering: a review. In: International Conference on Computational Science and Its Applications. Springer Cham International Publishing pp. 707–720. 2014.
Shyam R. Kumar S. Poornachandran P. and Soman K. P. (2015). Apache Spark a Big Data Analytics Platform for Smart Grid. Procedia Technology 21(2015) pp. 171–178.
Srinivas B. and Togiti B. (2015). Analysis of Mining on Big Data International Journal of Research and Computational Technology 7 pp. 1–10.
Stonebraker M. Abadi D. and DeWitt D. (2010). MapReduce and parallel DBMSs: friends or foes? Communications of the ACM 53(1) pp. 64–71.
Thakur B. and Mann M. (2014). Data Mining for Big Data: A Review. International Journal of Advanced Research in Computer Science and Software Engineering 4(5) pp. 469–473.
Ward J. S. and Barker A. (2013). Undefined by Data: A Survey of Big Data Definitions. arXiv preprint arXiv:1309.5821.
Wei C. Fadlullah Z. M. Kato N. and Stojmenovic I. (2014). On Optimally Reducing Power Loss in Micro-Grids with Power Storage Devices. IEEE Journal on Selected Areas in Communications 32(7) pp. 1361–1370.
Weilki J. (2013). Implementation of big data concept in organizations – possibilities impediments and challenges. In: Proceeding of 2013 Federated Conference on Computer Science and Information Systems IEEE pp. 985–989.
Wu X. Zhu X. Wu G. Q. and Ding W. (2014). Data Mining with Big Data. IEEE Transactions in Knowledge and Data Engineering 26(1) pp. 97–107.
Xhafa F. Naranjo V. and Caballe S. (2015). A software chain approach to big data stream processing and analytics. In: International Conference on Complex Intelligent and Software Intensive Systems Blumenau Brazil 8–10 July 2015 IEEE.
NIST. (2010). Office of the National Coordinator for Smart Grid Interoperability National Institute of Standard and Technology U.S. Department of Commerce “NIST Framework and Roadmap for Smart Grid Interoperability Standard Release 1.0” NIST Special Publication 1108 on the January 2010.
Yadav C. Wang S. and Kumar M. (2013). Algorithm and Approaches to Handle Large Data – A Survey. International Journal of Computer Science and Network 2(3) pp. 2277–5420.
Yan Y. Qian Y. Sharif H. and Tipper D. (2013). A Survey on Smart Grid Communication Infrastructures: Motivations Requirements and Challenges. IEEE Communications Surveys & Tutorials 15(1) pp. 5–20.
Zerhari B. Lahcen A. A. and Mouline S. (2015). Big data clustering: algorithms and challenge. In: Proceedings of the International Conference on Big Data Cloud and Applications (BDCA’15).
Zhang D. (2013). Inconsistencies in Big Data. In: Proceeding of IEEE international conference on Cognitive Informatics and Cognitive Computing IEEE.
Zikopoulos P. DeRoos D. Parasuraman K. Deutsch T. Giles J. and Corrigan D. (2013). Harness the Power of Big Data. McGraw-Hill.