Are you sure you are knitting the document and not just running individual code chunks? The eval option will only work when the document is knitted (Ctrl+Shift+K). If I run each chunk individually the chunk will run.
An alternative solution for this desired behavior would be to include the define the eval_condition parameter in the setup chunk. Perhaps this might solve your problem...