There are lots of things you can do without leaving your nice warm office, which is good for you but bad for The Department: anything you can do over the internet, someone else in on the other side of the world can do over the internet. Holes in their "Digital Armour" are important to fix quickly.
Take a look at their web site to see what you can learn, and poke at the server to see if you can get inside.
The Domain Name System converts a human-friendly web site name like donuts.govt.nz into a computer-friendly series of numbers that tells systems how to talk to each other. Sometimes there is a lot if useful information in a DNS record.
Perhaps you can learn something about The Department you could exploit later on. Or maybe their accounts have weak passwords you could guess, so you could take over their accounts?
Phishing is where you send a fake message that looks important to try and trick someone into clicking a link or opening a document that infects their computer.