Thesis Topic Details

Topic ID:
3563
Title:
Continuous Deployment for Big Data Analytics Applications
Supervisor:
Liming Zhu
Research Area:
Software Engineering, Cloud Computing, Distributed Systems
Associated Staff
Assessor:
Xiwei Xu
Topic Details
Status:
Active
Type:
R & D
Programs:
CS CE BINF SE
Group Suitable:
Yes
Industrial:
No
Pre-requisites:
--
Description:
Data scientists are increasingly moving from small-scale data analytics on a laptop to big data analytics in clusters (e.g. for genome analysis and financial data analysis). However, data scientists still need to perform explorative analytics development on their laptop or a small-scale environment and then deploy the data analytics application to clusters, often with extensive support from data product teams and engineering teams. If any issue arises during on the large-scale deployment or operation, data scientists need to revise their models back at their laptop and repeat the process again. Continuous Deployment/Delivery (CD) is a practice that copes with high frequency and automated deployment of applications. CD practices have been used in many types of applications but its use for data analytics applications and model development is still limited due to distinct model development cycle, data sampling and cluster deployment challenges. This project will expose you to data science and big data workflow. You will work within a team to develop new solutions to automate and simplify the workflow.

You will be working at NICTA (National ICT Australia)’s lab at Australia Technology Park, Sydney. The environment is exciting and gender-friendly including senior researchers, software engineers and other undergraduate/postgraduate students.

Comments:
- http://www.ssrg.nicta.com.au/projects/devops_book/ chapter 6.
- example big data software used: https://amplab.cs.berkeley.edu/software/
- email limingz@cse.unsw.edu.au
Past Student Reports
 
No Reports Available. Contact the supervisor for more information.

Check out all available reports in the CSE Thesis Report Library.

NOTE: only current CSE students can login to view and select reports to download.