Text in red colour was added/modified after the
- This is a 24h take-home exam.
- The exam runs from 12:00 (noon) on Tue, 2 Nov 2010 to
12:00 (noon) on Wed, 3 Nov 2010
- The basic exam question is available now, but the papers you
are asked to analyse will be made available only at 12:00 (noon) on Tue, 2 Nov 2010.
- The papers will be available electronically via this WWW page.
Email Gernot if you need hardcopies so we can try to
- The total exam is worth 35 marks.
- You will lose 3.5 marks for each hour, or part thereof, your
submission is late.
- You are not to get any help from anyone on the exam. You should
not talk to anyone else about the exam from the time you receive
the full details until you submit your solution.
- You have the choice of three different ways to submit your
- Submit a hardcopy by the deadline (12:00 (noon) on Wed, 3 Nov 2010). It must be accompanied by the signed
certification of sole
authorship. It must be submitted to Gernot Heiser,
Kevin Elphinstone, Aaron Carroll,
Bernard Blackham or Justin King-Lacroix in person. In
addition, you must submit an electronic version (PDF) within three
days of the end of the exam.
- Submit an electronic copy (PDF), via the give system, by the
deadline, and submit a hardcopy, including the signed certification of sole authorship,
within three days of the end of the exam.
- Submit, via the give system, by the deadline, a digitally
signed electronic copy (PDF). This must contain your full name,
student number, date, and the following declaration:
I hereby declare that this submission is my own work, and I have
not received any help whatsoever.
The file must be signed with PGP/GPG, be ASCII-armoured, and have an
extension .pdf.asc. This is
achieved (when using PGP 6.0 or later) with the command:
pgp -sa <file>
or, using GPG, with the command
gpg -sa <file>
See the GPG Example for how to use
GPG for signing and check that you've got it right.
Notes on electronic submissions
- In the cases where a paper submission is also required (cases 1 and
2 above), the electronic submission, when printed on a CSE printer, must
appear exactly as the submitted hardcopy.
- The submission must be in PDF format (and use the extension
- The submission must be made via the give system, mark
- If using give, check you can submit your exam (or any
exam report) well before the deadline. We will have little sympathy for
submission issues if you raise them five minutes before the deadline.
- Make sure that you only use Type-1 fonts, as others are
unprintable on some printers. LaTeX users can ensure the use of Type-1
fonts by producing a PostScript file with the command
dvips -Pwww -o file.ps file
and then converting this to PDF.
Notes on digitally signed submissions
Digitally signing your submission only makes sense if we can verify your
signature. I therefore require you to have your signature signed by
Gernot or Kevin beforehand, or within three days of the end of the
exam. Therefore, if you want to use the digital signature option, do the
- Familiarise yourself with PGP (or GPG). We will not
provide tutorials on this, it's up to you. Get yourself a public key
if you don't have one. Follow the recommended safeguards to keep
it secure. It will be like your normal signature!
- See Gernot or Kevin with your key and proof of identity. He
will then sign your key.
- This signed key can then be used to sign your exam. (Feel
free to get others to sign your key as well.) If you get your key
signed after the exam, make sure that it is the same key as used for
signing the exam.
- Make sure that PGP or GPG is installed on the system you are
going to use to write your exam, and that you can use it reliably. If
you stuff up, it's your own problem.
You are given two research papers (the links will be active from
12:00 (noon) on Tue, 2 Nov 2010):
- Paper 1 David,
Chan, Carlyle and Campbell: “CuriOS: Improving Reliability
through Operating System Structure”, 8th OSDI,
- Paper 2 Tang,
Mai and King: “Trust and Protection in the Illinois Browser
Operating System”, 9th OSDI,
You are to read, understand, and critically assess the papers. Questions
you may want to ask yourself for each of the papers:
These are only hints, I am not asking you to explicitly answer all these
for each paper. However, you may find those questions helpful in
critically analysing the papers. Imagine you are a reviewer for a
conference to which the papers have been submitted, and you are to judge
their contribution to the field.
- What problem is it trying to address?
- How well does it address the issue?
- How well do they motivate the value of solving the problem. Is
the problem a real problem? Is the problem/work significant?
- How does it relate to other work? Does
it reference relevant other work (as far as you can tell), does it
do the other work justice? Has it all been done before?
- How technically sound is it? Does their argumentation, the
presented data convince you? Should they have been looking at other
- How good are the results?
- How good/deep is their analysis?
- How easy would it be to reproduce their results?
- How general are their results? Can they be applied to other
systems? Did we learn some general truth?
Note that all papers are in fact published (and should therefore
meet certain quality standards one hopes :-). In order to get an idea of what program committees at
top systems conferences are looking for, have a look at this classic!
Here are sample solutions, which were done in “real-time”
by the students taking the exam (the two reports are from different students):
Note: this is an exam, not betting on horses (even if it's held on
Melbourne Cup day ;-). It is dangerous to guess what I might
think of the paper, or to guess that there'll be a good and a bad
one. Papers are selected on other criteria.
Sample report on David et
al. (1,300 words).
Other points the student could have made:
- There is no indication that the approach can deal with
shared state (and almost certainly it cannot). This is a pretty
serious limitation which put the value of the whole thing in
question (eg. how do you implement a file system?)
- The invocation model is exactly what was used in Mungi
more than ten years earlier
except that Mungi could deal with shared state and did it
about two orders of magnitude faster, on slower hardware!)
Note that the students were not expected to know this, as
Mungi and single-address-space OSes weren't covered in the
- Sample report on Tang et al. (840 words).
Cudos to the student for understanding the spinlock issue
which the authors didn't get! Other points the student could have made:
The split-driver approach, where most of the driver deals with
data only via opaque pointers and cannot see the actual data
it transfers, is neat. However, the paper completely fails to
investigates the implications (all drivers must be re-done,
this is much worse than the usual microkernel driver problem,
where driver re-use is actually possible, see ref. 
in the paper) and trade-offs (what does it do to
performance?) In fact, what does the scheme really buy, given
that the data is then sent unencrypted over the internet? Why
didn't they just use an IOMMU to encapsulate the drivers?
The percentage of vulnerabilities eliminated by this approach
seems surprisingly low, and one must ask whether this
justifies such a radical approach.
Was it really justified to put so much extra code into the
kernel? Why not stay with a proper microkernel approach and
build the extra functionality at user level?
What to submit
You are to submit a report which summarises for each paper the basic
ideas behind their work. You are to give a critique of the technical
merits, achievements and shortcomings (if any). The papers are not
directly related, so you don't have to compare them.
I am intentionally not specifying a length limit. However, I
strongly encourage you to be concise. Lengthy submissions will almost
certainly be unfocussed and waffly. I cannot imagine a decent job in
excess of 3000 words, and would imagine that a very good submission
would stay well below 2000 words total. If your report gets longer than
this you should step back and try to focus.
A good way to structure your review is the standard approach taken
by conference program committees, which tend to use some variant of a
basic structure which has the following sections:
Summary of the paper. This is about three paragraphs summarising
what the paper i trying to achieve and how it goes about it, and
how relevant the work is. Given that you're trying to convince me
that you got it, you may go into a bit more detail than the
typical reviewer would (who is selected as an expert in the field
and doesn't have to prove themselves). So you may want to write up
to a page here (but be concise!)
Also, this section should not focus on
criticising the paper (although with a bad paper I frequently
find it difficult to state what they are doing without noting
that it is wrong...)
Pros: What you like about the paper (list of bullet points or
Cons: What you don't like about the paper.
Criticism of the work. Things which are wrong, insufficient,
could be improved. But also detailed discussion of the
strengths. If you were a reviewer whose job is to recommend
whether the paper should be accepted or rejected, this is where
you make your case. But even if you generally like the paper,
discuss its shortcomings, and even if you would reject it, discuss
its redeeming features. This is the most important part of the review.
Minor issues that should be fixed (typos, grammar, etc). Things of
secondary importance that are nevertheless worthwhile fixing. This
section is less important for you (as the paper is already
published) but if you have some other things that annoy you,
here's the place to get rid of them.
Questions to the authors. Many conferences have a rebuttal period,
where authors get the chance to comment on the reviews before the
decision is made. If you think there is something the authors
could clarify to help you make a decision, this is the
place. Again, given that the paper is already published, this
section is obviously optional.
Points that must be addressed if accepted. Many conferences use a
shepherding process, where accepted papers are assigned a shepherd
who supervises the revisions and ensure that the authors follow
the requests of the reviewers. Again, in your case it's too late,
but if you think that there are improvements that should
have been made prior to publication, then this is the place to
make your point.
Note: In order to help us to perform an unbiased assessment of
your report, we would appreciate if you do not put your name on
the report itself, only your student ID. Of course, your name
must appear on the certificate that is attached to the report. However,
as long as this certificate is on a separate page, we can assess the
reports without looking at names.
What I will be looking for
You will be marked on the level of understanding and critical analysis
portrayed in you submission. All relative to what can be reasonably
expected from you (I know that none of you have a PhD in OS yet :-)
You may find it useful to look at the
or 1999 exams,
and the sample reports provided there.
10 Nov 2010.