https://support.microsoft.com/en-us/topic/description-of-the-security-update-for-microsoft-exchange-server-2019-2016-and-2013-march-2-2021-kb5000871-9800a6bb-0a21-4ee7-b9da-fa85b3e1d23b

 

  1. Update fails to stop all services ( do this manually ) 
  2. Exchange services might remain in a disabled state after you install this security update. This condition does not indicate that the update is not installed correctly. This condition might occur if the service control scripts experience a problem when they try to return Exchange services to their usual state.

    To fix this issue, use Services Manager to restore the startup type to Automatic, and then start the affected Exchange services manually. To avoid this issue, run the security update at an elevated command prompt. For more information about how to open an elevated Command Prompt window, see Start a Command Prompt as an Administrator.

GD Star Rating
loading...
GD Star Rating
loading...

 

Error: he following error was generated when “Serror.Clear(); $dlIFile =join-path $RolelnstallPath “bin ExSMIME.dII”; $regsvr =join-path (join-path $env:SystemRoot system32) regsvr32.exe; start-SetupProcess -Name:”$regsvr” -Args:”/s -Timeout:120000; “was run: “Microsoft.Exchange.Configuration.Tasks.TaskException: Process execution failed with exit code 3. at Microsoft.Exchange.Management.Tasks.RunProcessBase.InternalProcessRecordo at Microsoft.Exchange.Configuration.Tasks.Task.

Microsoft.Exchange.Configuration.Tasks.Task.InvokeRet ableFunc(Strin • funcName, Action func, Boolean terminatePipelinelfFailed)”

 

Fix

Checking logs there was missing Visual C++ 2012 and 2013 runtime. 2013 was a new requirement for CU18/19 so no surprises there, but 2012 has been a req for as far back as I can see. Definitely wasn’t present. So the pre-flight dependency check script doesn’t actually check for deps… From what I can tell it checks for the presence of RSAT and that’s about it. Once that was sorted (I had to install both 2012 & 2013) I was able to run CU19 again successfully. 

GD Star Rating
loading...
GD Star Rating
loading...
Partial mitigation for clients unable to patch immediately:
 
 
Microsoft has released a script to assist in checking for signs of being compromised by the recent exchange vulnerabilities:
 
 
Nmap Check

There is a method to check whether a recently patched (or unknown) server is vulnerable to the SSRF exploit.

Please run this procedure for each of your assigned clients, either that have been or not patched, we MUST ensure they are not vulnerable, even if we think we applied the patch. 

  • Jump on a util machine inside customer network, or whatever machine as long as it is internal.
  • Download nmap from nmap website and install with default settings
  • Download nse script from Microsoft https://github.com/microsoft/CSS-Exchange/releases/latest/download/http-vuln-cve2021-26855.nse
  • Move nse script file just download under c:\program files (386)\nmap\scripts
  • Open nmap
  • In the Command filed type: nmap -sV -p 443 –script=http-vuln-cve2021-26855 -script-args vulns.showall IPOFTHEEXCHANGESERVER

You must confirm it says “NOT VULNERABLE”

Example of vulnerable server:

 

What to do if your compromised

  • Reset of all users’ account. ALL of them. Service accounts and administrator included.
  • Review of all new users added/remove/edited during the last 2 weeks as well as security group change made.
  • Immediate isolation of Exchange server (If server is exploited, full access is possible). Creation of a fresh Exchange, migrate mailboxes off the old Exchange (for this I invoke the Exchange experts). Burn the old Exchange server.
  • Restore from Backup

 

If you install the patch by downloading the patch and just double clicking on it, the patch will install but not fix the vulnerability because exchange services are still running, and it can’t replace the files.

 

See the known issues section. This also has a known side effect of leaving some services disabled.

https://support.microsoft.com/en-us/topic/description-of-the-security-update-for-microsoft-exchange-server-2019-2016-and-2013-march-2-2021-kb5000871-9800a6bb-0a21-4ee7-b9da-fa85b3e1d23b

 

 

Microsoft released patches for older/unsupported Exchange CU’s to help customers securing their servers faster:

 

https://techcommunity.microsoft.com/t5/exchange-team-blog/march-2021-exchange-server-security-updates-for-older-cumulative/ba-p/2192020

 

 

Microsoft on Monday released a one-click mitigation software that applies all the necessary countermeasures to secure vulnerable environments against the ongoing widespread ProxyLogon Exchange Server cyberattacks.

Called Exchange On-premises Mitigation Tool (EOMT), the PowerShell-based script serves to mitigate against current known attacks using CVE-2021-26855, scan the Exchange Server using the Microsoft Safety Scanner for any deployed web shells, and attempt to remediate the detected compromises.

“This new tool is designed as an interim mitigation for customers who are unfamiliar with the patch/update process or who have not yet applied the on-premises Exchange security update,” Microsoft said.

The development comes in the wake of indiscriminate attacks against unpatched Exchange Servers across the world by more than ten advanced persistent threat actors — most of the government-backed cyberespionage groups — to plant backdoors, coin miners, and ransomware, with the release of proof-of-concept (PoC) fueling the hacking spree even further.

Based on telemetry from RiskIQ, 317,269 out of 400,000 on-premises Exchange Servers globally have been patched as of March 12, with the U.S., Germany, Great Britain, France, and Italy leading the countries with vulnerable servers.

Additionally, the U.S. Cybersecurity and Infrastructure Security Agency (CISA) has updated its guidance to detail as many as seven variants of the China Chopper web shell that are being leveraged by malicious actors.

GD Star Rating
loading...
GD Star Rating
loading...
When it tries to boot and fail , click Menu Button
 
Admin ( Password default : 6633222 )
 
Press 1 on the dial pad for Network Settings.
Press 6 on the dial pad for Advanced Settings
Press 1 on the dial pad for LAN Port Settings (VLAN for the voice
packets only)
Press 2 on the dial pad for VLAN Mode
Press 2 enable the VLAN for the voice packets
Press 3 on the dial pad for VLAN ID
Enter a valid VLAN ID of 1-1401
 
Save Settings reboot
GD Star Rating
loading...
GD Star Rating
loading...

From Microsoft:

“There is a Known Issue that halts the installation progress of the February 9, 2021 security update (KB4601318). To address this issue, we have released a new servicing stack update (SSU), KB5001078. You must install this new SSU before installing the February 9, 2021 security update.”

 

This caused me some frustration previously, with the stuck update that would never, ever finish – even after a reset of the windows update components…
Have verified the order of installation seems to work

KB5001078 was already installed on the server but still the update would not install (stuck at downloading 0% on one server and preparing to install on another server).

 

I had to do what Microsoft published on their article as the previous SSU (KB4601332) was already installed on the server before KB5001078 was installed:

 

https://support.microsoft.com/en-us/topic/february-9-2021-kb4601318-os-build-14393-4225-c5e3de6c-e3e6-ffb5-6197-48b9ce16446e

 

To mitigate this issue on devices that have already installed KB4601392 and are not making progress installing KB4601318, restart your device and then follow only steps 1, 2 and 4a from Reset Windows Update components manually. Then restart your device again.

 

GD Star Rating
loading...
GD Star Rating
loading...

Mailboxes in 365 were not receiving email , this was caused by the recent Exchange 2010 security patches being installed – Update Rollup 32 for Exchange Server 2010 Service Pack 3 (KB5000978)

 

  1. Get Exchange to the Latest CU
  2. Windows Server 2008 SP2

    Ensure the latest Windows updates are applied, this must include:

    • KB4019276 to add TLS 1.2 capability as a default secure protocol for SChannel.
    • KB3161949 for the current version of WinHTTP.

    Install 3154517 for .NET Framework 3.5.1.

    Windows Server 2008 R2 SP1

    Ensure the latest Windows updates are applied, this must include:

    • KB3161949 for the current version of WinHTTP.
    • KB3080079 to add TLS 1.2 capability to RDS (optional).

    Install 3154517 for .NET Framework 3.5.1.

    Windows Server 2012

    Ensure the latest Windows updates are applied, this must include:

    • KB3161949 for the current version of WinHTTP.

    Install 3154517 for .NET Framework 3.5.1.

3)

When the above steps have been completed, registry settings need to be added to Enable TLS 1.2 for SChannel, and Enable TLS 1.2 for .NET 3.5.

TLS 1.2 for SChannel

Import the following:

Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2]
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Client]
"DisabledByDefault"=dword:00000000
"Enabled"=dword:00000001
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\TLS 1.2\Server]
"DisabledByDefault"=dword:00000000
"Enabled"=dword:00000001

.NET 3.5

Import the following:

Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework\v2.0.50727]
"SystemDefaultTlsVersions"=dword:00000001
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\.NETFramework\v2.0.50727]
"SystemDefaultTlsVersions"=dword:00000001
GD Star Rating
loading...
GD Star Rating
loading...

Qnap Thin pool repair

Stop all of services and umount all of volumes

Step1: Stop all of services

/etc/init.d/services.sh stop

/etc/init.d/qsnapman.sh stop

/sbin/daemon_mgr lvmetad stop “/sbin/lvmetad” rm /var/run/lvm/lvmetad.socket

Step2: Confirm which volume group or pool need to repair

pvs # list all of volume groups

lvs -a # list all of volumespools and volume groups

lvs -o+time # can list the created date/time of volumespools lvs -o+thin_id # can list the devices id of volumes

Check Volume Groups using “pvs” command

PV VG Fmt Attr PSize PFree

/dev/drbd1 vg1 lvm2 a– 7.24t 0 # /dev/drbd1 indicates md1(RAID group 1)

/dev/drbd2 vg2 lvm2 a– 7.26t 0 # /dev/drbd2 indicates md1(RAID group 2)

.

.

.

# One Volume Group may include >= 2 RAID groups

Check devices on the volume group/pool which we need to repair using “lvs -a” command

LV

VG

Attr

LSize

Pool Origin

Data% Meta%

Move Log Cpy%Sync

Convert

      

lv1

vg1

Vwi-aot—

6.00t

tp1

100.00

# Data volume 1

lv1312

vg1

-wi-ao—-

756.00m

# snapshots pool

  

lv2

vg1

Vwi-aot— 200.00g tp1

100.00

# Data volume 2

lv544

vg1

-wi——- 74.14g

# Reserved to repairing temporary

 

snap10001

vg1

Vwi-aot—

6.00t tp1

lv1

100.00

# Snapshot volume

snap10002

vg1

Vwi-aot—

6.00t tp1

lv1

100.00

# Snapshot volume

.

      

.

      

.

      

Step3: Assume vg1 need to repair, we need to umount the volumes which belong to the vg1(lv1 will be mounted in /share/CACHEDEV1_DATA….. and so on….)

If there is any volumes or snapshots which is mounted and the mounted volumes belong to the volume group which need to repair, please umount them

# below umount all of data volumes umount /share/CACHEDEV1_DATA

umount /share/CACHEDEV2_DATA

.

.

umount /share/CACHEDEV${#}_DATA

# below umount snapshots umount /dev/mapper/vg1-snap*

if can not umount, lsof /share/CACHEDEV{#}_DATA to check, and “kill -9” these process. Try to umount the volumes again.

Remove all of cache devices and inactivate volume group

Step1: You can list which cache devices on the pool:

ls -l /dev/mapper/ # will list all of devices on the pool

Step2: The result of the above command looks like below(below assume vg1 is the volume group which need to repair)

brw——-

1

admin

administrators

253,

9

2020-02-15

10:16

cachedev1

brw——-

1

admin

administrators

253,

50

2020-02-15

10:16

cachedev2

crw——-

1

admin

administrators

10,

236

2020-02-15

18:14

control

brw——-

1

admin

administrators

253,

7

2020-02-15

10:16

vg1-lv1

brw——-

1

admin

administrators

253,

8

2020-02-15

10:16

vg1-lv1312

brw——-

1

admin

administrators

253,

10

2020-02-15

10:16

vg1-lv2

brw——-

1

admin

administrators

253,

11

2020-02-18

01:00

vg1-snap10001

brw——-

1

admin

administrators

253,

13

2020-02-19

01:00

vg1-snap10002

brw——-

1

admin

administrators

253,

15

2020-02-20

01:00

vg1-snap10003

brw——-

1

admin

administrators

253,

17

2020-02-21

01:00

vg1-snap10004

brw——-

1

admin

administrators

253,

19

2020-02-22

01:00

vg1-snap10005

.

        

.

        

.

        

Step3: Find cachedev${#} and remove them using dmsetup

dmsetup remove cachedev1 dmsetup remove cachedev2

.

.

.

dmsetup remove cachedev${#}

Step4: Inactivate the volume group which need to repair

lvchange -an vg1 # Assume vg1 is the volume group which need to repair

Step5: Please check again with “ls -l /dev/mapper” command to confirm there is no any block device of vg1. The result should be the below:

crw——- 1 admin administrators 10, 236 2020-02-15 18:14 control

Collect logs of thin pool metadata and backup metadata of the thin pool

Step1: Download collect tools

wget http://download.qnap.com/Storage/tsd/utility/tp_collect.sh

Step2: Execute collect tools to collect logs of the metadata or backup metadata of the pool

sh tp_collect.sh

When the tp_collect.sh start running, please remember enter “pool id”. The pool id is the id of the pool which we want to repair. For examples: vg1/tp1, please input 1vg2/tp2, please input 2……and so on…..

If execute tp_collect fail, you need to let the customer plug one USB external drive(about 100G or more) in the NAS to backup metadata

# assume vg1/tp1 need to collect or repair lvchange -ay vg1/tp1_tmeta

# Change directory to the USB external device cd /share/external/DEVXXXXXX

# backup metadata using dd command

dd if=/dev/mapper/vg1-tp1_tmeta of=tmeta.bin bs=1M

If tp_collect is executed completely or successfully, you can skip the above flow and please backup collect.tgz.

Step 3: Please confirm the backup of metadata is correct

# A. thin check original metadata

pdata_tools_8192(or 4096) thin_check /dev/mapper/vg1-tp1_tmeta

# B. thin check backup metadata

pdata_tools_8192(or 4096) thin_check /mnt/collect/tmeta.bin # If the metadata is backup in the USB external drive, please pdata_tools_8192(or 4096) thin_check /share/external/DEVXXXXXX/tmeta.bin

The result of the above A, B need to be the same. If the A, B is the same, Please be sure to backup the tmeta.bin to another storage or USB

external drive!!! If the vg{#}-tp{#} repaired fail, the vg{#}-tp{#}_tmeta can be restored using tmeta.bin.

 

GD Star Rating
loading...
GD Star Rating
loading...

Check metadata:

Please use thin_check to check the metadata of the pool to confirm what wrongs in the metadata.

pdata_tools_8192( or 4096) thin_check /dev/mapper/vg1-tp1_tmeta –non

# Assume the ghost snapshot happen in vg1/tp1. Please according to happened ghost snapshot pool to adjust vg${#}

-tp${#}_tmetadata

# if the firmware version of QTS is less than 4.3.5, please remove “–non” option

Generally, the metadata has the below wrong:

If the metadata has no such as above wrong, please contact sustain team to analyze.

Backup metadata before repairing the metadata:

Caution!!! This is very important task!!!

Please must make a backup of the metadata to avoid the repairing task fail.

If the repairing task fail, the storage may crash. We can use the backup metadata to restore the storage.

Please use the below guide to make the backup of metadata:

https://drive.google.com/file/d/14HSVV6DWSnqc7FXTSUukzJWB812Q8Tkv/view?usp=sharing

Repairing flow and limitations:

Limitations:

If the storage configuration and QTS firmware versions can not meet the following condition, please contact sustain team to do repairing task.

1. The QTS firmware version is great or equal than 4.3.5.

2. The chunk size of the metadata need to be 8192. Please use the below command to confirm.

echo $((16#`hexdump -s 340 -n 2 /dev/mapper/vg1-tp1_tmeta | awk ‘NR==1{print $2}’`*512))

Repairing steps:

Step 1: Download “pdata_tools” which support thin_rebuild option from the below link and transfer new “pdata_tools” to the NAS.

# Download

wget http://download.qnap.com/Storage/tsd/utility/pdata_tools_X86_64.zip # X86_64 version wget http://download.qnap.com/Storage/tsd/utility/pdata_tools_ARM_64.zip # ARM_64 version

# Unzip

unzip pdata_tools_X86_64.zip unzip pdata_tools_ARM_64.zip

# Please rename to pdata_tools

mv pdata_tools_X86_64 pdata_tools mv pdata_tools_ARM_64 pdata_tools

Step 2: Change “pdata_tools” to executable file.

chmod +x pdata_tools

Step 3: Run pdata_tools with thin_rebuild option.

./pdata_tools thin_rebuild /dev/mapper/vg1-tp1_tmeta # This process will execute about couple minutes or hours. Please wait for the process done patiently.

Step 4: Use thin_check to check metadata again to confirm the “reference counts” wrongs are all fixed. If the result of thin_check still has wrongs, please contact sustain team to do the repairing task.

./pdata_tools thin_check –clear-needs-check-flag /dev/mapper/vg1-tp1_tmeta

Step 5: Restore the storage:

lvchange -ay vg1 # bring up the volume group storage_util –sys_startup_p2 # mount all of volumes

/sbin/daemon_mgr lvmetad start “/sbin/lvmetad” # enable lvm cache

/etc/init.d/services.sh start # start up all of services

 

GD Star Rating
loading...
GD Star Rating
loading...

Recently was trying to disable access to a subfolder on an IIS site. Adding IP Restrictions to the site , and looking the Logs , the address of the requestor was coming up as the Gateway of the DMZ.

We need to use the X-forwarder-For header for this , however a need a WAF or application level firewall to do this stamping for us

After enabling Proxy Mode on Cloudflare for the hostname , the correct IP address starting populating for X-forwarder-For header so I could turn on blocking

GD Star Rating
loading...
GD Star Rating
loading...
  1. I recommending backing up the Autocomplete from both Accounts via Backing up and restoring the AutoComplete cache of Outlook 2010, 2013, 2016, 2019 and Office 365 – MSOutlook.info
  2. Login to account you would like to copy the autocomplete from in Outlook , it will create a C:\Users\%username%\AppData\Local\Microsoft\Outlook\RoamCache\Stream_Autocomplete_0_XXXXXXXXXX1.dat file
  3. You can use nk2 editor to read this if you want to confirm http://www.nirsoft.net/utils/outlook_nk2_edit.html 
  4. Login to the account you would like to copy the to , it will create a C:\Users\%username%\AppData\Local\Microsoft\Outlook\RoamCache\Stream_Autocomplete_0_XXXXXXXXXX2.dat file
  5. Cloud outlook and rename Stream_Autocomplete_0_XXXXXXXXXX2.dat to Stream_Autocomplete_0_XXXXXXXXXX2.datold and rename Stream_Autocomplete_0_XXXXXXXXXX1.dat to Stream_Autocomplete_0_XXXXXXXXXX2.dat. 
  6. Reopen outlook
  7. Add a new contact that’s not in your autocomplete e.g. test@test.com and send an email
  8. Close Outlook and reopen it
  9. It should merge your old Autocomplete with your existing one
GD Star Rating
loading...
GD Star Rating
loading...