Steve's Evolution Payroll Blog

Evolution and the CVE-2019-0708 (Remote Desktop) vulnerability

Today, Microsoft released a set of fixes for a serious vulnerability in Remote Desktop (née Terminal Services), where a remote unauthenticated user can run code on the server as system, and this is a big deal. BIG BIG DEAL.

The always-reliable Brian Krebs reports on it here, and Microsoft's writeup is here.

The purpose of this post is not to talk about the vulnerability, but how it impacts the administrator of an Evolution system.

Short description: Server 2003, Server 2008 and Server 2008 R2 are all vulnerable and must be patched.

Server 2012 and newer are safe.

This vulnerability has nothing to do with Evolution itself, and customers hosted on Asure's AWS infrastructure are not impacted.

Note: Please distinguish between Service Packs (SP1, SP2), which are simply collections of regular updates for an operating system, and "R2" releases of Windows: the latter are different operating systems (e.g., "Server 2008" and "Server 2008 R2" are different systems).

Server 2008 and Server 2008 R2 can be updated via routine Windows Updates, they are included in the monthly rollups for May 2019 (both the standard and the security-only).

Special pain with Server 2003

Server 2003 requires substantial special handling; it doesn't appear to be availble on Windows Update, so the fix must be downloaded directly from Microsoft. I found this link to the Windows Update catalog for Server 2003; choose the patch that applies to your server.

The second complication for Server 2003 is that Internet Explorer on that platform does not appear to support a sufficiently new (and secure) SSL protocol to talk to Microsoft's download servers. You'll need to either use a different (but still probably old and insecure) browser on the 2003 machine, or fetch it from a more modern system and transfer the file over the network. However you get it, get it there.

Yes, I still have customers running Server 2003 in production.

Did I do it right?

It's also important to know how to check: did my server actually receive the critical update? You can find out in Windows Update history by looking for these KB (knowledge base) numbers:

• KB4500331; manually-installed patch for Server 2003
• KB4499149; monthly rollup for Server 2008
• KB4499180; security-only rollup for Server 2008
• KB4499164; monthly rollup for Server 2008 R2
• KB4499175; security-only rollup for Server 2008 R2

If your update history shows the KB update for your operating system, it's good. None of these updates has impacted the Evolution modifier/license, so you can install them without that concern.

Please aggressively check all your servers, not just the ones running Evolution: if a server is exploited (from the inside or the outside), it has the real possibility of causing real havoc in your network.

What about the desktop?

Modern desktop operating systems (Windows 8/8.1 and Windows 10) are not impacted by this, but Windows 7 and XP Service Pack 3 are. Win7 gets the same update as Server 2003 R2, and XP/SP3 gets the Server 2003 update.

I didn't see any mention of Windows Vista, so you may be out of luck.

And if you're still on Window 95, you're not vulnerable :-)

Best Practices for Replacing Server Hard Drives

Many of my customers still run older Evolution servers, but are not refreshing the hardware due to the impending migration to the Asure Evolution cloud.

The most common hardware failures, by far, are the hard drives, and with lapsed warranties, I typically send my customers to third-party sellers of spare drives to avoid paying through the nose with Dell. I have been very happy with ServerSupply.com, but customers have used eBay as well.

These drives only have to last another year or so; I've had very good experience with refurbished drives, no need to purchase a brand new one.

Important: if ordering a spare drive for a Dell server, it's important to look for "Dell OEM" in the product description; a stock drive from Seagate or Western Digital or whatever may not work in some Dell RAID controllers unless it has Dell-specific firmware. You can try a stock drive, but I don't recommend it unless you're sure you don't have one of the brand-enforcing RAID controllers.

In any case, I usually recommend that customers get more than one drive so they're covered in advance for the inevitable next failure (especially for the Linux database servers that use 6 or 8 identical drives), and that they follow this procedure for doing the physical drive replacement. This mostly applies to warranty replacements too.

• You'll need a screwdriver, a Sharpie marker, and (if you bought a second spare drive) some blank Post-It notes.

• Be really sure you have identified the right server; I've had a customer pull a good drive from a good server because the machines weren't marked properly.

• Be really sure you have identified the failed drive; it's usually lit with a red or amber light, and if I have told you a drive has failed, I'll have told you the slot that it's in. Note: drive slots are usually usually numbered from 0, not 1.

• Pull the failed drive from the machine — this can be done while it's running, that's why we have RAID — and immediately put a Post-It note on it with a big black X in Sharpie marker: it's surprisingly easy to get confused about which drive is which. Put the bad drive aside.

• On the replacement drive, write in Sharpie "Installed <DATE>" on the label in case there is ever a question about the history of this drive.

• Remove the old drive from the carrier (that's what the screwdriver is for), insert the replacement drive into the carrier. Pay attention to the orientation so it goes in the same way as the one that came out. It's not easy to get them backwards, but I'm sure it's possible.

• Insert the new drive back into the server, insuring it's firmly seated. It should start rebuilding the array immediately with a lot of drive activity.

  Note: if the drive lights don't come on within a few seconds, it's not in right.

• If you bought a spare drive, write "Spare for <SERVER>" on the Post-It, stick it on the spare, and put it near the machine. You will thank yourself in six months when another drive fails and you'll know right away whether the drive in your hand works for that machine.

• If I'm your consultant, let me know you did this so I can keep an eye on the array rebuild remotely.

• Shred or otherwise dispose of the failed drive

  Note: if this drive was replaced under warranty, almost all vendors require that you return the bad drive with the provided label and box. Be sure to do this to avoid being charged for it.