Utilize este identificador para referenciar este registo:
https://hdl.handle.net/1822/86367
Título: | A data-locality-aware distributed learning system |
Autor(es): | Carneiro, Davide Oliveira, Filipe Novais, Paulo |
Data: | 2022 |
Editora: | Springer, Cham |
Revista: | Lecture Notes in Networks and Systems |
Citação: | Carneiro, D., Oliveira, F., Novais, P. (2022). A Data-Locality-Aware Distributed Learning System. In: Novais, P., Carneiro, J., Chamoso, P. (eds) Ambient Intelligence – Software and Applications – 12th International Symposium on Ambient Intelligence. ISAmI 2021. Lecture Notes in Networks and Systems, vol 483. Springer, Cham. https://doi.org/10.1007/978-3-031-06894-2_6 |
Resumo(s): | Machine Learning problems are significantly growing in complexity, either due to an increase in the volume of data, to new forms of data, or due to the change of data over time. This poses new challenges that are both technical and scientific. In this paper we propose a Distributed Learning System that runs on top of a Hadoop cluster, leveraging its native functionalities. It is guided by the principle of data locality. Data are distributed across the cluster, so models are also distributed and trained in parallel. Models are thus seen as Ensembles of base models, and predictions are made by combining the predictions of the base models. Moreover, models are replicated and distributed across the cluster, so that multiple nodes can answer requests. This results in a system that is both resilient and with high availability. |
Tipo: | Artigo em ata de conferência |
URI: | https://hdl.handle.net/1822/86367 |
ISBN: | 978-3-031-06893-5 |
e-ISBN: | 978-3-031-06894-2 |
DOI: | 10.1007/978-3-031-06894-2_6 |
ISSN: | 2367-3370 |
Versão da editora: | https://link.springer.com/chapter/10.1007/978-3-031-06894-2_6 |
Arbitragem científica: | yes |
Acesso: | Acesso restrito UMinho |
Aparece nas coleções: |
Ficheiros deste registo:
Ficheiro | Descrição | Tamanho | Formato | |
---|---|---|---|---|
ISAmI_Carneiro.pdf Acesso restrito! | 347,61 kB | Adobe PDF | Ver/Abrir |