Course Surveys 2011
Gernot's Comments on CATEI Survey
This was impressively positive and at least as good as previous
years. However, with a return ratio
of 47%, not too much can be read into this.
Nice to see all the positive comments.
The improvement suggestions all appear in my detailed survey, I will
comment on them there.
Gernot's Comments on LiC Survey
Many thanks to all students for taking the time to answer the course
survey. This year we achieved a 95% return rate!
The results mostly speak for themselves. The course seems to be in good
shape, overall satsifaction was high.
The biggest gripe (by a huge margin) was the three-hour
lecture block. This
is a real problem to change, but I accept that it is a major pain
for students. I'll see whether something can be done about it, but
I'm not overly optimistic, given that we're at the mercy of
centralised time tabling.
Interestingly, the biggest gripe of previous years, documentation,
has all but gone away, despite moving from a commercial platform
back to a research system! This is a compliment to the NICTA folks
who produced the docs.
The question about whether om was a good thing or not had
more or less the expected response: Yes, it would be good to get
closer to the hardware, and yes, the project is hard enough as it
is. Still, it is a shame that you don't get to deal with caps and
many of the seL4 abstractions.
We'll definitely re-consider this
point for next year, and see whether we can provide some more
minimalist library without making the project harder. For this
year I'm glad we did what we did, as changing the whole setup is
always a big risk. In the end, all groups got a working OS
together (and almost all on time), I consider that a big success!
The exam was also considered hard. I think this year it was a bit
on the hard side, as both were SOSP papers (which are among the
longest around and tend to have no faults that are easy to
spot). However, I hadn't seen anything else recently that was a
good fit to the course. Hopefully, next time is better.
Having said that, people need to understand that this is an elite
course and you can't expect things to be easy. It focusses on
providing insights, and this is what the exam tests. And it's the
reason why having passed AOS is known to help in the job market!
Not too much lecture time on seL4 specifics, and milestones not
too well specified: These are features! We want you to
work things out for yourselves, including how exactly the kernel
ticks, and what the best design is (and that includes forcing you
However, we will examine whether there is a mismatch between
milestone specs and tutor expectations, if so we'll fix that. (But
also notice that very rarely were demo marks docked, so I don't
think this is a big issue.)
No tutor access during week-ends and milestones due on Monday:
Fair point, I'll consider having milestone demos due on Tue.
Chairs in seminar room: This is a complaint (by a
minority of students) which had been made last year already (but
not in the previous 4 years we've been using that room). Maybe
this is an indirect consequence of the 3-hour lecture block...
Relevant one-off comments:
Not enough emphasis on research paper critical reading and
evaluation during the semester
We listed 1—3 relevant papers for each week's lecture, so
you were clearly directed to relevant literature. I also always
encourage people to read these papers, but inevitably hardly
anyone does. I can't read them for you, so what do you expect us
What we can do in the future is to provide the relevant papers a
week or so before the lecture, in the (probably vain) hope that
someone will read them. Worth trying...
Some of the specifications were far too vague, and were
assessed on criteria that were not revealed to the
It is a core feature of this course that we do not spoon-feed
students. We intentionally provide only rough guidance, combined
with extensive feedback by tutors. We consider this a central part
of the course experience.
As far as assessment goes: No-one's milestone demo got marked down
because they chose a non-ideal design or implementation, but the
tutors might have told them why their choice was bad and advised
them to change it. This is very different from assessing based on
undisclosed criteria, and something we will not change!
Some more exciting drivers would be nice.
I agree, but hard to fit in without making the project even
harder. Also, the platform is limited in that it only provides
trivial and complex devices, not much in between.
With possibly a new hardware
platform next year, we will revisit this point.
Auto marking scripts are helpful to resolve conceptual
Providing you with a test harness and test cases would only work
if the milestones were defined much more narrowly. We do not want
this, as constraining the project too much would compromise the
learning experience as it would drastically limit your design
space, and avoid having to think through all the trade-offs. This
is core systems work.
Furthermore, testing your system is your job, and when thinking
about your design you should think about how to test it. Taking
this off you would be counter-productive!
Too much microkernels? Yes, the course is very biased towards
microkernels, but that is design. Discussion about why they failed
commercially? Well, in fact, they didn't! The majority of embedded
multitasking systems are microkernel-based, and seL4 is likely to
change things further in favour of microkernels.
Too much overlap with normal OS: This comment pops up every now
and then. I guess there are parts of the system which are similar
to what you've done before, but would it be better if we supplied
big slabs of code? (We all agree that we supply too much OM...)
Om bugs: True, that's the risk of a new project. Should be better
Material such as audio, internals of other OSes, etc: Yes, would
be nice to have, but we have limited time, and there's no
consensus on what to drop. Also, what exactly we offer depends on
the expertise we have in house, which varies (and we presently
don't have a Windows or audio-driver expert). We really want to
ensure that all the lecturers are highly competent in what they teach!
All up, that was a lot of useful feedback. Thanks to all of you!
08 Dec 2011.