Okay, the quickest way to get on my bad side is to somehow screw up our domain’s Domino Directory, names.nsf. For the uninitiated, names.nsf holds all of the configuration information for your entire environment. It holds specific server info, domain-wide settings, as well as all of your users and their authentication info. Needless to say, you don’t fuck with names.nsf if you don’t know what you are doing.
Well at my new job we have normal NT admins on the coasts that have full admin rights to our Notes servers. It was like that when I got here, so being the new guy, I haven’t pushed hard to change that. After today, it’s changing, mark my words.
You see, an admin in Atlanta accidentally replaced the design of the Domino Directory with the design of the MAIL TEMPLATE. Yes, the mail template. And the bad thing is that he had no clue that he did it. He said he was just replacing the design on a user’s mailbox.
You can imagine what this did to the environment. I had another admin in Chicago ask “Is there anything wrong with the mail server?” I said I didn’t think so, and then I looked over his shoulder. He had the Notes console up and the errors were streaming off the page. It basically said it couldn’t find the certifier for our domain, repeatedly. After my heart started again, I ran for my machine.
I opened our NAB and got email instead. You could visibly see the veins on my neck pop out. I checked the database and yup, the template was showing iNotes6. I hastily replaced the design whilst praying to various deities and alternately cursing the bastard responsible. Once the design was replaced, there were still several errors related to on-disk structure, so I rebooted the Domino services. Luckily everything was back to normality at that point.
That’s when forensics began, and we traced it to Atlanta. In the meantime, the design replace had replicated to EVERY server in my domain (eleven to be exact) So I had to replace the design and reboot each one to get the environment right. I’m glad this happened first thing in the morning. It made it so our west coast offices didn’t even know there was problem.