On-Line Survey 2009

Survey ID 1285 Title COMP9242 09 Description Course Evaluation Survey for COMP9242 Advanced Operating Systems. Version for Session 2, 2009. Anonymous Yes Fill Ratio 81% (17/21) # Filled 17 # Suspended 1 # Not Filled 3

indicates required field

Your comments will help us to assess and improve our courses, not only for future generations, but for your further study in CS&E. We really look at the results and appreciate your feedback! Several changes to the course over the years were a direct result of student feedback. And, as always, we'll publish the uncensored results on the course web site. Note: Please do not enter "no comment" or something similar into comment boxes. If you don't have anything to say, just leave the box empty.

1. Quick Evaluation

1. Give a high rating if you have a good opinion of something (e.g. interesting, useful, well-structured, etc.). Give a low rating if you have a bad opinion of something (e.g. too slow, confusing, disorganised, etc.)   Question type : Single answer -- Radio Button   Excellent Satisfactory Poor Gernot Heiser 14 (82%) 3 (18%) 0 (0%) 0 (0%) 0 (0%) Kevin Elphinstone 14 (82%) 3 (18%) 0 (0%) 0 (0%) 0 (0%) Guest lecturer Leonid Ryzhyk 7 (41%) 5 (29%) 4 (24%) 1 (6%) 0 (0%) Guest lecturer Peter Chubb 7 (41%) 6 (35%) 4 (24%) 0 (0%) 0 (0%) Guest lecturer Godfrey van der Linden 8 (47%) 5 (29%) 4 (24%) 0 (0%) 0 (0%) Tutors/demonstrators 15 (88%) 2 (12%) 0 (0%) 0 (0%) 0 (0%) Course web pages 7 (41%) 5 (29%) 5 (29%) 0 (0%) 0 (0%) Exam 8 (47%) 6 (35%) 3 (18%) 0 (0%) 0 (0%) Reference material 5 (29%) 1 (6%) 9 (53%) 1 (6%) 1 (6%) Computing resources 9 (53%) 3 (18%) 4 (24%) 1 (6%) 0 (0%) COMP9242 overall 14 (82%) 2 (12%) 1 (6%) 0 (0%) 0 (0%)

2. General

2. Which factors most influenced your decision to enrol in this course?   Question type : Multiple answer -- Check Box Interest in operating systems as an area of study 14 (82%) Chance to build a system 10 (59%) Chance to get fingers really dirty 6 (35%) Would like to do some systems research 8 (47%) Looking for a challenge 13 (76%) Looking for an easy course 2 (12%) Friends told me it was good 12 (71%)

3. Other factors not mentioned above? Question type : Short-answer Answer at the bottom page (4 comments)

4. Would you recommend this course to another student such as yourself?   Question type : Single answer -- Radio Button Yes 17 (100%) No 0 (0%)

5. The course is heavy on design and implementation issues. It also tries to remain close to present research issues. What do you think about this?   Question type : Single answer -- Radio Button   Too much Just right Too little Theory/general principles 0 (0%) 2 (12%) 14 (82%) 0 (0%) 1 (6%) OS design and implementation 1 (6%) 2 (12%) 11 (65%) 3 (18%) 0 (0%) Current research issues 0 (0%) 1 (6%) 11 (65%) 5 (29%) 0 (0%)

6. What were the best things about this course? Question type : Long-answer Answer at the bottom page (13 comments)

7. What were the worst things about this course? Question type : Long-answer Answer at the bottom page (13 comments)

8. How does the workload in this course compare to workloads in other ...   Question type : Single answer -- Radio Button   Much Lighter Similar Much Heavier COMP courses at this level 0 (0%) 0 (0%) 2 (12%) 4 (24%) 11 (65%) COMP courses in general 0 (0%) 0 (0%) 1 (6%) 2 (12%) 14 (82%) Courses in general 0 (0%) 0 (0%) 2 (12%) 2 (12%) 13 (76%)

9. How does the overall quality/value of this course compare to other ...   Question type : Single answer -- Radio Button   Among the best Average Among the worst COMP courses at this level 12 (71%) 4 (24%) 0 (0%) 1 (6%) 0 (0%) COMP courses in general 13 (76%) 3 (18%) 0 (0%) 1 (6%) 0 (0%) courses in general 13 (76%) 3 (18%) 0 (0%) 1 (6%) 0 (0%)

10. What background knowledge do you think you were missing that would have helped you in this course? Is distinction in COMP3231/9201 a suitable preparation? Is it too harsh? Question type : Short-answer Answer at the bottom page (7 comments)

3. Content/Syllabus

11. Please rate the relevance/appropriateness of the lecture topics.   Question type : Single answer -- Radio Button   Very relevant Average Inappropriate N/A L4 general and L4 API 15 (88%) 1 (6%) 1 (6%) 0 (0%) 0 (0%) 0 (0%) Microkernels in general 12 (71%) 4 (24%) 1 (6%) 0 (0%) 0 (0%) 0 (0%) Threads and Events 14 (82%) 2 (12%) 1 (6%) 0 (0%) 0 (0%) 0 (0%) Caching & TLBs 10 (59%) 4 (24%) 3 (18%) 0 (0%) 0 (0%) 0 (0%) Virtual machines 7 (41%) 5 (29%) 5 (29%) 0 (0%) 0 (0%) 0 (0%) Computer security 10 (59%) 3 (18%) 3 (18%) 0 (0%) 1 (6%) 0 (0%) Performance Evaluation 10 (59%) 5 (29%) 1 (6%) 1 (6%) 0 (0%) 0 (0%) Device Drivers 8 (47%) 3 (18%) 5 (29%) 0 (0%) 0 (0%) 1 (6%) Towards trustworthy systems 7 (41%) 3 (18%) 4 (24%) 2 (12%) 0 (0%) 1 (6%) OS X Audio 5 (29%) 5 (29%) 4 (24%) 1 (6%) 0 (0%) 2 (12%) Scalability 8 (47%) 3 (18%) 3 (18%) 1 (6%) 0 (0%) 2 (12%) I/O scheduling 7 (41%) 4 (24%) 5 (29%) 0 (0%) 0 (0%) 1 (6%)

12. Please tell us how interesting you found the lecture topics.   Question type : Single answer -- Radio Button   Very interesting Ok Boooooring! L4 general and L4 API 5 (29%) 7 (41%) 3 (18%) 2 (12%) 0 (0%) Microkernels in general 7 (41%) 7 (41%) 3 (18%) 0 (0%) 0 (0%) Threads and Events 7 (41%) 8 (47%) 2 (12%) 0 (0%) 0 (0%) Caching & TLBs 7 (41%) 8 (47%) 1 (6%) 1 (6%) 0 (0%) Virtual machines 10 (59%) 4 (24%) 1 (6%) 2 (12%) 0 (0%) Computer security 8 (47%) 3 (18%) 4 (24%) 1 (6%) 1 (6%) Performance Evaluation 5 (29%) 4 (24%) 7 (41%) 1 (6%) 0 (0%) Device Drivers 7 (41%) 4 (24%) 3 (18%) 3 (18%) 0 (0%) Towards trustworthy systems 6 (35%) 4 (24%) 4 (24%) 3 (18%) 0 (0%) OS X Audio 9 (53%) 4 (24%) 4 (24%) 0 (0%) 0 (0%) Scalability 5 (29%) 4 (24%) 8 (47%) 0 (0%) 0 (0%) I/O scheduling 5 (29%) 8 (47%) 4 (24%) 0 (0%) 0 (0%)

13. Which material do you think will be most useful to you in the future?   Question type : Long-answer Answer at the bottom page (17 comments)

14. Which material, not currently in this course, would you liked to have seen covered? Question type : Long-answer Answer at the bottom page (4 comments)

15. Which of the current topics would you like to see scaled back or excluded? Question type : Long-answer Answer at the bottom page (4 comments)

4. Lectures

16. What factors caused you to attend lectures?   Question type : Multiple answer -- Check Box I had enough spare time 8 (47%) The lectures were too good to miss 12 (71%) Given the pace and lack of a textbook, I could not afford to miss the lectures 8 (47%) It was as good a place as any to take a nap 0 (0%) I wanted to be seen to be there 3 (18%) None, I skipped most 1 (6%)

17. What were the reasons for skipping lectures?   Question type : Multiple answer -- Check Box Overall workload in this and other courses 3 (18%) Lecture notes and references cover the material adequately 1 (6%) Lectures are boring 0 (0%) There was not enough material to justify attending lectures 0 (0%) First half of the course was more interesting than second half 0 (0%) None, I attended (almost) all 14 (82%)

18. Any suggestions for improving lectures? Question type : Long-answer Answer at the bottom page (8 comments)

5. Project

19. What was the level of difficulty various parts of the project?   Question type : Single answer -- Radio Button   Too easy Just right Too hard Milestone 0 3 (18%) 1 (6%) 10 (59%) 3 (18%) 0 (0%) Milestone 1 1 (6%) 5 (29%) 10 (59%) 1 (6%) 0 (0%) Milestone 2 0 (0%) 3 (18%) 9 (53%) 5 (29%) 0 (0%) Milestone 3 0 (0%) 1 (6%) 14 (82%) 2 (12%) 0 (0%) Milestone 4 0 (0%) 0 (0%) 14 (82%) 3 (18%) 0 (0%) Milestone 5 1 (6%) 1 (6%) 12 (71%) 3 (18%) 0 (0%) Milestone 6 0 (0%) 1 (6%) 7 (41%) 7 (41%) 2 (12%) Milestone 7 0 (0%) 2 (12%) 9 (53%) 6 (35%) 0 (0%) Milestone 8 1 (6%) 1 (6%) 8 (47%) 7 (41%) 0 (0%) System documentation 0 (0%) 1 (6%) 12 (71%) 2 (12%) 2 (12%) Project overall 1 (6%) 0 (0%) 14 (82%) 2 (12%) 0 (0%)

20. How well was the project specified?   Question type : Single answer -- Radio Button   Very clear Ok Confusing Milestone 0 6 (35%) 1 (6%) 6 (35%) 2 (12%) 2 (12%) Milestone 1 8 (47%) 2 (12%) 6 (35%) 1 (6%) 0 (0%) Milestone 2 8 (47%) 3 (18%) 5 (29%) 0 (0%) 1 (6%) Milestone 3 7 (41%) 2 (12%) 7 (41%) 1 (6%) 0 (0%) Milestone 4 7 (41%) 1 (6%) 8 (47%) 1 (6%) 0 (0%) Milestone 5 6 (35%) 2 (12%) 6 (35%) 3 (18%) 0 (0%) Milestone 6 7 (41%) 1 (6%) 5 (29%) 4 (24%) 0 (0%) Milestone 7 6 (35%) 2 (12%) 6 (35%) 3 (18%) 0 (0%) Milestone 8 6 (35%) 1 (6%) 6 (35%) 4 (24%) 0 (0%) System documentation 3 (18%) 2 (12%) 5 (29%) 6 (35%) 1 (6%) Project overall 4 (24%) 4 (24%) 7 (41%) 2 (12%) 0 (0%)

21. What was the quality of...   Question type : Single answer -- Radio Button   Excellent Ok Poor Documentation/reference material 0 (0%) 2 (12%) 7 (41%) 6 (35%) 2 (12%) Supplied code 0 (0%) 4 (24%) 9 (53%) 4 (24%) 0 (0%) Hardware platform 8 (47%) 5 (29%) 3 (18%) 1 (6%) 0 (0%) Consultation time help/support 11 (65%) 4 (24%) 2 (12%) 0 (0%) 0 (0%) On-line help/support 7 (41%) 7 (41%) 1 (6%) 2 (12%) 0 (0%)

22. Any suggestions for improving the project? Question type : Long-answer Answer at the bottom page (9 comments)

6. Anything Else

23. Any other comments/suggestions that might help us to improve the course in the future? Question type : Long-answer Answer at the bottom page (5 comments)

3. Other factors not mentioned above? 1: Kevin and everyone else at ERTOS told me I should. 2: Needed a new shirt 3: Wanted to prove my ability (not being a cse student) 4: prestige? it was prescribed for me, but I would have taken it anyway 6. What were the best things about this course? 1: Being able to not only build but also design an operating system from scratch 2: Deep learning about hardware and operating systems principles. 3: Having to do almost everything myself 4: Heavily indepth special topic lectures. I'd rather get to know a handful of topics in alot of depth 5: Intense. Full-on. 6: Interesting lectures 7: Project, exam, and lectures were interesting and fun. 8: The awesome project, which involved a whole lot of programming. The exam was fairly interesting, too. 9: The experience gained working with microkernels and OS implementation in general 10: The lectures were very interesting 11: You get to see your OS develop over time. 12: a good opportunity to build a complex system 13: the chance to design an operating system with microkernel implemented. 7. What were the worst things about this course? 1: A little too much redundancy with extended operating systems. 2: A lot of time was spent debugging the project. You don't learn much while debugging, just that you are prone to making stupid mistakes. 3: Amount of time needed to work on milestones 4: Debugging and debugging. 5: I can't think of anything bad at the moment. maybe the labs weren't long enough to get everybody's demos done. 6: I had to return my Slug. NOW how am I supposed to run my OS? 7: Not being able to rm -rf my os in week 12 when I realised it sucked 8: The long nights sometimes spent in the labs when something wasn't quite working. 9: The same as the best things? Well, the lack of sleep it causes. Documentation is really bad esp. for L4. 10: The sometimes very vague documentation and such, meaning that things such as setting up a system or finding out odd little things about random sections of code sometimes seemed to be based more on luck than skill. Mind, with enough time applied it always worked out. 11: The workload was really heavy and in many cases there was a disconnect between lectures and what we were doing, so it almost seemed as if there were two parallel courses, rather than one unified course. 12: Writing documentation as the final milestone. 13: libserial 10. What background knowledge do you think you were missing that would have helped you in this course? Is distinction in COMP3231/9201 a suitable preparation? Is it too harsh? 1: 75 in COMP3231 is too low, a COMP9242 group with two people like that would probably fail the course. You should require 80 or 85 in COMP3231 to make the groupwork fairer 2: Despite taking the course without the prerequisites I was able to pick up various concepts along the way, including caching, ARM assembly, page table structures. I found the lectures to be at about the right level for me. 3: It's sufficient as a minimum, but it may be worth putting up recommended background such as 3151 and 3121 4: None 5: Seemed okay. 6: not sure, this is the first OS course I've taken at UNSW. 7: yes it is suitable 13. Which material do you think will be most useful to you in the future?   1: All of it. I found the threads and events lecture particularly awesome. The caching lecture will be very useful. 2: An even greater understanding of how Operating Systems achieve common tasks, and hence an ability to write code taking better advantage of the underlying OS, as well as perhaps as a basis for further OS or device driver coding in future. 3: Caching, virtual machines 4: Caching, virtual machines, performance evaluation 5: General idea about what techniques are involved in OS development 6: General understanding of how an operating system works. Good for writing efficient code. 7: Knowledge of L4 and microkernel deesign, security and scalability. 8: Microkernels (including L4), computer security, trustworthy systmes. 9: No idea. Sorry. 10: Overall knowledge of how L4 and microkernels in general work. 11: Performance Evaluation lecture 12: The caching & TLBs, Threads and Events, Device Drivers and Virtual machines lectures. 13: The information that can help improve normal programming performance such as caching and threads. 14: Virtual machine and OS X audio would be most useful for me 15: Virtual machines, micro kernels in general 16: microkernels in general, caching & TLDs, virtual machines, computer security. 17: most of it will be useful 14. Which material, not currently in this course, would you liked to have seen covered? 1: It would have been good to have more of a look at more commonly used monolithic kernel designs - not just the basics, but more how they got past common / interesting / difficult issues etc. 2: More on computer architecture (I'm avoiding the proper comp arch course because it isn't supposed to be very good) 3: Resource management (e.g. Solaris Zones). Perhaps being introduced to a larger variety of OS design ideas (both at the kernel level and user-level APIs), such as more about RTOSs, operating systems for scientific/grid computing, creative APIs, "experimental" operating systems. 4: The design or internals of other OSes for comparison. I hear Microsoft is making available the source code of their research kernel. It would be good to have a look at this. http://www.microsoft.com/resources/sharedsource/licensing/basics/wrklicense.mspx 15. Which of the current topics would you like to see scaled back or excluded? 1: Large chunks of the Security and Towards Trustworthy Systems lectuers weren't interesting, in particular the talk about various security models sent me completely to sleep, the parts about seL4 were quite interesting though. 2: Performance Evaluation 3: Scalability ...just kidding 4: Well, everything covered seemed to be pretty decent. There's not enough time, really, in a semester to cover this course in a way that would satisfy everyone - taking this into account a very good job has been done. Some topics that perhaps had more emphasis in the past would have had to have some of that removed with the shortening of semesters, and if some of that was content I'd prefer more than someone else, then that's just the way it is. 18. Any suggestions for improving lectures? 1: Have an hour gap between the lab and the lecture 2: Have more points jotted down in the lecture notes. 3: Hold the lectures closer to the CSE labs. The lecture in the Australian School of Business was awesome, what a cool lecture theatre, I would have loved to have every lecture in there. The closer the lecture theatre is to the CSE labs, the less time is wasted getting there. If you get marked at 1:55pm and start packing up then, there's no way to get to NICTA until about 2:10 or 2:15pm. If you insist in having lectures at NICTA, it would be nice to have desks (to make it easier to take notes), and to abandon the electronic whiteboard (because it is completely unreadable) 4: I was doing all nights for the milestones the night before. Thursday was one of the only nights my friend and I could meet up. Made it hard to stay awake all lecture. Don't put lecture on same day milestones are due would make this better for next time. 5: It would be nice not to have them the same day the milestones were due. This led to all too much napping in them, and large amounts of caffeine consumption. The coffee shop at NICTA would have made a fair bit off of us. 6: Overall the lectures were good, though towards the second half, the relevance to us became almost minimal, since we couldn't really take anything from them that we would be using in the massive project consuming all our time. Somehow meshing what we were implementing with what we were learning about would have been interesting, though maybe impossible. 7: Perhaps a bit more student involvement; for instance asking the students to read a paper once every few weeks and having a brief discussion on it in the lecture. 8: no, I think the lectures were very good overall... I missed Peter Chubb's lecture though. 22. Any suggestions for improving the project? 1: Better documentation for all areas would be nice. 2: Better documentation, clearer spec. 3: Pay someone to rewrite libserial. 4: Perhaps find somewhere to slip in some low-level programming (bonus feature?). Something involving ARM assembly and/or modifications to the microkernel. More documentation for the BUILD SYSTEM would be much appreciated. E.g. how to change optimisation level, how to see the gcc/ld command lines, how to change linker scripts. 5: Please make the supporting libraries we are given thread safe. Also more different bonus tasks would be fun. 6: There are some notes on the main page which provide some additional information about certain milestones which should probably be added to those milestones' pages 7: Upgrading to a recent OKL4 version would be nice, since we would (hopefully) not have to wonder if a bug that is manifesting is ours or the kernel's. 8: Write some more "official" tests to help everyone test their code. Presumably you have to write these tests anyway, to mark the project at the end. It would be nice to share them with students, and would help clarify the spec. Otherwise it is just a total panic getting marked on Fridays, as you have no idea whether you have missed something important, and if you have, you might have to implement it in 20min and try and get marked again before the lecture starts. 9: make the labs longer or earlier than 1 hour before the lecture to accommodate them running over time make the move to seL4? 23. Any other comments/suggestions that might help us to improve the course in the future? 1: In my opinion, the more you put into this course, the more you get out of it. A couple of my fellow students were not motivated or had a heavy workload from other subjects, and I think more should be done to discourage such students as they face a lot of stress with considerably less gain than students who put a lot into the course. 2: In this survey, too many of the fields are compulsory, more fields should have an N/A option. There are some things that I don't really want to rate, e.g. I shouldn't rate the quality of a lecturer if I fell asleep in their lecture due to working on the project all night, but the survey forces me to pick a rating... ? 3: Just a note, we haven't gotten feedback on the documentation or the exam, so our perception of how easy/hard it was may not be right. 4: Lectures seem to assume additional background other than COMP3231, such as history in OS, recent developments which I could not relate. Also ensure COMP3231 has the system call? assignment (Wasn't done when I did it). 5: Maybe if the L4 programming lecture at the start gave a few more practical, project specific hints.