Email

If you send an email it is a good idea to tell the user a way to monitor the network themselves. Eg. Software solutions like SCOM or WhatsUp Gold.

Include a "To myself". It gives visibility to others who are interested in what needs to be done to fix the problem and makes
it easier to remember to send the 'done' email. E.g. "done - CRM is alive again". 

Example:

To: SSWALL

Hi All,

Here is the summary of the outage plan:

Planned/Unplanned:	Planned
Change Description:	Install Windows Updates and Restart Server
Risk (see table below):	LOW RISK (LOW Probability and MEDIUM Impact)
Reason For Change:	Windows 2016 Windows Updates
Uptime over last month:	91.361%
Planned Outage (mins):	150
Planned Start Time:	26 October 9:00 PM
Planned Finish Time:	26 October 11:30 PM
Affected Services:	\\Windows Server 2016
	http://sharepoint.ssw.com.au http://intranet.ssw.com.au http://projects.ssw.com.au

Risk Lookup Table by Probability and Impact:

Risk
		Probability
		Low	Medium	High	Unknown
Impact	Low	Low risk	Low Risk	Low Risk	Medium Risk
	Medium	Low Risk	Medium Risk	Medium Risk	High Risk
	High	Medium Risk	High Risk	High Risk	High Risk
	Unknown	Medium Risk	High Risk	High Risk	High Risk

Figure: Clearly showing the potential risks

Note: The following servers will be affected

http://wug.ssw.com.au/

To myself,

To show others who are interested in what needs to be done to fix the problem:
Detailed Change Plan:
1) Lockout users via IIS
2) Backup server
3) Install Windows Updates 
4) Reboot server
5) Follow test plan
6) Based on result of test plan, follow backout plan if procedure failed
7) Procedure completed

Test Plan:
1) Check Event log for errors
2) Check each affected service is running
3) Call test users to start “Test Please” on the affect services 
4) Get result of user “Test Please” by email by 11:15 PM

Backout Plan:
1) Restore server from backup

Note: <This is as per rule What is your server reboot/restart policy? >

Immediately before the scheduled downtime, check for logged in users, file access, and database connections.

Users

Open 'Windows Task Manager' (Run > taskmgr) and select the 'Users' tab. Check with users if they have active connections, then have them log off.

Figure: Connected users can be viewed in Task Manager

Files

Open 'Computer Management' (Run > compmgmt.msc), then 'System Tools > Shared Folders'. Check 'Session' and 'Open Files' for user connections.

Figure: Computer Management 'Open Files' View

Database

Open SQL Server Management Studio on the server. Connect to the local SQL Server. Expand 'Management' and double-click 'Activity Manager'.

Figure: SQL Management Studio 'Active Connections' View

Once these have been checked for active users, and users have logged off, maintenance can be carried out.

Restarts should only be performed during the following time periods

1. Between 7am and 7:05am
2. Between 1pm and 1:05pm
   
3. Between 7pm and 7:05pm
   

If a scheduled shutdown is required, use the PsShutdown utility from Microsoft's Sys Internals page.

Always reply 'Done' when you finish the task. ​

When passwords have to be changed they must meet the following minimum requirements:

• Not contain all or part of the user's account name
• Be at least six characters in length
• Contain characters from three of the following four categories:
  
• English uppercase characters (A through Z)
• English lowercase characters (a through z)
• Base 10 digits (0 through 9)
  
• Non-alphanumeric characters (e.g., !, $, #, %)
  

​Every 180 days clients will be required to change their password, they can change it when:

• Login to their ​computer
  
• Terminal server to another computer
• VPN

This allows users to change their password by making a VPN connection to the office.

We also enforce a lockout policy so if a user gets their password wrong 5 times, their account will be locked out for 15 minutes.​

If you want to change your password sooner, press [ctrl] [alt] [delete] then click "Change Password" button.

​Hardware Outage:

• Firewall
• Switch
• Blade Servers
• SAN Storage
• UPS 

• Active Directory Domain Services
• O365 Services; Teams, SharePoint, Exchange, OneDrive
• File Servers
• SQL Servers
• IIS Servers

Determining What Services are Disrupted

At SSW we use WhatsUp Gold for our device monitoring, however, there are many tools for this, Solarwinds, SCOM, etc

1. Login to monitoring service
2. Check to see what services are down

First Contact

1. What services have been disrupted?
2. What is the impact of these services?
3. Is an email to everyone in your company required?
4. What are your next steps?

What if you cannot reach anyone?​​

If you cannot reach anyone move onto the Email section.​​

Email​​

If from the previous discussion you have determined that an email needs to be sent to your entire company, or you have decided this is necessary if you cannot contact anyone above, send an email in the following format: 

To: ALL
Subject: Outage Notice
 
Hey All,
 
We are experiencing an outage and the following services have been affected:
·       Xxx
·       Yyy
·       Zzz
 
We are working on restoring these services and will keep you updated.
 
Thank you,
SysAdmins

A separate email needs to be sent to SysAdmins outlining what was discussed on the call. If no one was contactable, please proceed with what you have determined on your own.

To: SysAdmins
Subject: SysAdmins – Outage Notice
 
Hey team,

As per our conversation.
 
The following services are disrupted:
·       Xxx
·       Yyy
·       Zzz
 
The impact of these services disrupted are:
·       Xxx
·       Yyy
·       Zzz
We have decided that an email to ALL is/is not required.
 
To myself,
 
The next steps to resolving this are:
 
1.      Xxx
2.      Yyy
3.      Zzz
 
Thank you,
SysAdmin

Next Steps did *not* resolve the issue

If you have completed your tasks but the issue has not resolved, please try to make contact with the SysAdmin team again and send an updated To Myself ​​email.

Next Steps resolved the issue​​

If your actions have resolved the issue, please notify ALL of the services being restored and update your To Myself email.

What is Multi-Factor Authentication (MFA)?

MFA is another layer of security for your users and administrators, it adds another 'password' or code that you can receive in a device that you possess - a phone, for example - to make it more difficult for attackers to steal your account.
If they guess or brute-force your password, they still need the second code to make it to your account.

Generally, every time you log in on a service, it will ask for your normal password and an additional code. This code can be retrieved through an authenticator app, through an SMS, or even through a phone call to your mobile.

It is best practice to apply MFA to your Administrators first, as their accounts are the most important on the company and have access to all resources, and your users second, which still benefits from added security.

If you want to let your users reset their own, on-premises passwords directly from the cloud, you need to have Password Writeback enabled in Azure AD Connect!

You can read more about Password Writeback from the Microsoft Documentation: https://docs.microsoft.com/en-us/azure/active-directory/authentication/howto-sspr-writeback

When setting up Azure AD Connect, you need to set the "Password Writeback" option:

If you get your UPS to email you when an event occurs then you will have answers to these questions.
The problem is that there is no uniform software that will work with all UPS's as they all have their own format.
All UPS's come with management software that can perform these actions. You just need to install it.

We use an MGE UPS so we use Personal Solution Pac which allows you to run script files on events. We just call a script file which will send us an email.

​

Figure: Who created this file?

Figure: Terminal into your file server using Terminal Services

Figure: It was Jatin!

The easiest way is to configure Windows file auditing.

Thankfully, Windows Server come with built-in file auditing. Any changes create and delete can be logged to your system event log. Here's how to set it up.

How to implement auditing on your file server

1. Terminal Server into the file server
2. In Windows Explorer, locate the directory you want to configure logging for (e.g. C:\Inetpub\wwwroot for logging changes to your website files)
3. Select Security tab | Advanced

   

   Figure: Select the folder you want to configure auditing for

4. Click the Auditing tab
   
5. Select the users whose usage you want to monitor (usually all users, so select Everyone)

   

   Figure: Select Everyone so that anyone who modifies any of the files will be logged
   

6. Select what you want to monitor. For best performance, we only tick the options in shown in the figure below - there's no need to log when someone opens a file.

   

   Figure: Select these 4 options (only audit the events you need to audit - there's no need to log when someone opens a file)

7. Click OK and OK again to apply the changes. The process may take some time depending on the number of subfolders and files selected.
   Now you need to configure the system event log.
   
8. Open Control Panel->Administrative Tools->Event Viewer
9. Right-click the Security node and Control Panel | Administrative Tools | Event Viewer
10. Right-click the sure Overwrite events as needed is checked

    

    Figure: Keep your log file to about 250MB - otherwise, your system performance may suffer
    

Checking who created the file

Now test to see if auditing is working.

1. On the server, create a file called "test.aspx" somewhere in the path that is being audited
2. Open Control Panel->Administrative Tools->Event Viewer
3. Select the Security node, and notice the entries that have been created. They will have a similar format to the figure below.

   

   Figure: Any creates, deletes and updates now get logged to the Event Log

That's all! It is also great for finding out who accidentally deleted files from the file system.

Furthermore, we can dump the event log to an Access or SQL Server database to make it easier to handle. Here is how to do it:

• Download the scripts: one for Access database and the other for SQL Server.
• Find and change the strEventDBConn variable to your connection string, also, modify strEventDB and tblEvents variable to your database name and table name.
• Write down the names of the servers to monitor in EventHosts.txt.

Done, now you need only double-click to start it.

Figure: Caught an action on remote server and logged it to database

This script is originally from http://pubs.logicalexpressions.com/pub0009/LPMArticle.asp?ID=340.​​

​

Figure: Bad example - Keeping a database is unnecessary

Figure: Good example - Using Certify The Web​

Joining your company's domain is a trade-off. If you join the domain, the company is the one responsible for managing your device, so all company rules and policies will be applied to it (Windows Update frequency, users, password resets, etc) and you will need to go through your SysAdmins if you have troubles with it.

If you choose to not join the domain, the machine management is all yours, giving you more freedom on the machine, but any automatic scripts would need to be done manually.

Below some pros and cons of joining the domain:

Figure: Bad example - an email is sent on completion

Figure: Good example - a record is logged on completion

​​​Now you are able to be aware of missing backups. You can make automatically notification based on above table e.g. by SQL Reporting Services data-driven subscription

​​​​​​

Figure: Good example - We can easily see the uptime of all our servers

WEP, No SSID broadcast, allowed MAC addresses are all OK but these are more home security.

For the office, you need something a bit more robust and not requiring much management overhead.

It is recommended to use Radius authentication to integrate with your Active Directory.

When you create a website, you can only access it through HTTP (http://website), and not securely through HTTPS if you do not own an SSL Certificate.

When it comes to website certificates, you can choose from free or paid SSL certificates!

Free certificates can be obtained from Certificate Authorities like Let's Encrypt, which is helping provide free and automated certificates for the web.

1. Free certificates provide the same level of SSL encryption as paid certificates;
2. Free certificates provide HTTPS with a green padlock on the address bar of your browser, just like paid certificates;
3. Free certificates can be automatically renewed, easily.

 

1. Paid certificates gives you warranty against misuse or wrongly issued certificates;
2. Paid certificates are normally valid for at least 1 year or more, while free certificates are only valid for 3 months;
3. Paid certificates offer support for any errors or problems you have with your certificates.

Good Example: Comodo Paid Certificate Authority

SSL Certificates are an important part of any reputable website, so if you are operating a small website, blog, testing environment, personal site, anything that doesn't need too much support, getting a free certificate is the way to go.

If your business or site does not fit on the above affirmation, getting a paid certificate is the best option!

​As a rule, you should never use a user account for accessing systems, reports, tasks and other long-running applications that do not need human or user interaction to run.

Service accounts provide a security context where the applications run, without the need to worry about their passwords or privileges. If a user changes their password, you will not break anything, because service account password normally does not expire and changing them is never needed.

Also, if the security of a user account is breached, you do not have to worry about any other systems being compromised - that account was not being used to run any applications. Always keep your service accounts passwords safe and complex, and you will never need to worry about it.​

1. You will run out of space - which means you will have to copy or move old (but still used) setup files around to other drives (\\bee\d$\SetupOld\ ) or other machines e.g. \\tuna\SetupFiles. This fragmentation of your setup files can cause confusion for your users.
2. When you retire or rename the old server, links to the old server location will not work

So how do you get around this problem? The answer is in the Distributed File System (DFS). Instead of having several server-specific file share locations, you can have a domain-wide setup location that offers a seamless experience to your users. DFS will even track a history of when and where file locations were moved.

Figure: The Distributed File System consolidates many separate file shares into one convenient location for your users

How to have a company-wide Word template:

• Modify your Normal.dotm file to have the headings and format that you want for Word document
• Create standard employee email footer files e.g. JamesZhou.htm or JamesZhou.txt
• Put the files on a network location - this is the place that will have the master copies 
  
• ​Have a logon script which is set up through Group policy that will copy the file to the users' computer when they logon.
  e.g. a PowerShell login script like https://github.com/SSWConsulting/SSWSysAdmins.LoginScript