Energy-aware cluster reconfiguration algorithm for the big data analytics platform spark

Duan, K; Fong, S; Song, W; Vasilakos, AV; Wong, R

Energy-aware cluster reconfiguration algorithm for the big data analytics platform spark

Duan, K Fong, S Song, W Vasilakos, AV Wong, R

Permalink

Publisher:: MDPI AG
Publication Type:: Journal Article
Citation:: Sustainability, 2017, 9, (12), pp. 1-13
Issue Date:: 2017-12-18

Open Access

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is open access.

Adobe PDF

Download full textAdobe PDF (1.88 MB)

View on publisher's site

View statistics

Full metadata record

Field	Value	Language
dc.contributor.author	Duan, K
dc.contributor.author	Fong, S
dc.contributor.author	Song, W
dc.contributor.author	Vasilakos, AV
dc.contributor.author	Wong, R
dc.date.accessioned	2022-08-23T01:03:50Z
dc.date.available	2022-08-23T01:03:50Z
dc.date.issued	2017-12-18
dc.identifier.citation	Sustainability, 2017, 9, (12), pp. 1-13
dc.identifier.issn	2071-1050
dc.identifier.issn	2071-1050
dc.identifier.uri	http://hdl.handle.net/10453/160711
dc.description.abstract	The development of Cloud computing and data analytics technologies has made it possible to process big data faster. Distributed computing schemes, for instance, can help to reduce the time required for data analysis and thus enhance its efficiency. However, fewer researchers have paid attention to the problem of the high-energy consumption of the cluster, placing a heavy burden on the environment, especially when the number of nodes is extremely large. As a consequence, the principle of sustainable development is violated. Considering this problem, this paper proposes an approach that can be applied to remove less-efficient nodes or to migrate over-utilized nodes of the cluster so as to adjust the load of the cluster properly and thereby achieve the goal of energy conservation. Furthermore, in order to testify the performance of the proposed methodology, we present the simulation results implemented by using CloudSim.
dc.language	English
dc.publisher	MDPI AG
dc.relation.ispartof	Sustainability
dc.relation.isbasedon	10.3390/su9122357
dc.rights	info:eu-repo/semantics/openAccess
dc.subject	12 Built Environment and Design
dc.title	Energy-aware cluster reconfiguration algorithm for the big data analytics platform spark
dc.type	Journal Article
utslib.citation.volume	9
utslib.for	12 Built Environment and Design
pubs.organisational-group	/University of Technology Sydney
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	/University of Technology Sydney/Faculty of Engineering and Information Technology/School of Electrical and Data Engineering
utslib.copyright.status	open_access	*
pubs.consider-herdc	false
dc.date.updated	2022-08-23T01:03:49Z
pubs.issue	12
pubs.publication-status	Published
pubs.volume	9
utslib.citation.issue	12

Abstract:

The development of Cloud computing and data analytics technologies has made it possible to process big data faster. Distributed computing schemes, for instance, can help to reduce the time required for data analysis and thus enhance its efficiency. However, fewer researchers have paid attention to the problem of the high-energy consumption of the cluster, placing a heavy burden on the environment, especially when the number of nodes is extremely large. As a consequence, the principle of sustainable development is violated. Considering this problem, this paper proposes an approach that can be applied to remove less-efficient nodes or to migrate over-utilized nodes of the cluster so as to adjust the load of the cluster properly and thereby achieve the goal of energy conservation. Furthermore, in order to testify the performance of the proposed methodology, we present the simulation results implemented by using CloudSim.

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/160711