Posts Tagged ‘veeam’

 Report % Rate of change from the last Backup Job Run

Add-PSSnapin -Name VeeamPSSnapIn -ErrorAction SilentlyContinue

Disconnect-VBRServer | out-null

connect-vbrserver -server localhost

$JobsOutput = @()

Foreach ($JobObject in Get-VBRJob | ?{$_.JobType -eq "Backup"})

{

$LastSession = $JobObject.FindLastSession()

$ChangeRate = ($LastSession.Info.Progress.TransferedSize/$LastSession.Info.Progress.TotalUsedSize)*100

$JobOutput = New-Object -TypeName PSObject

$JobOutput | Add-Member -Name "Jobname" -MemberType Noteproperty -Value $JobObject.Name

$JobOutput | Add-Member -Name "Endtime" -MemberType Noteproperty -Value $LastSession.endtime

$JobOutput | Add-Member -Name "TotalUsedSize" -MemberType Noteproperty -Value $LastSession.Info.Progress.TotalUsedSize

$JobOutput | Add-Member -Name "ReadSize" -MemberType Noteproperty -Value $LastSession.Info.Progress.ReadSize

$JobOutput | Add-Member -Name "TransferedSize" -MemberType Noteproperty -Value $LastSession.Info.Progress.TransferedSize

$JobOutput | Add-Member -Name "ChangeRate" -MemberType Noteproperty -Value $ChangeRate

$JobsOutput += $JobOutput

}

$JobsOutput | Out-GridView

Disconnect-VBRServer | out-null

However one result is not a fair estimate of rate of change , and I don’t have a Veeam One license so I decided to Average Stuff

You can run on One VM or all the VM’s in a Backup Job

Add-PSSnapin -Name VeeamPSSnapIn -ErrorAction SilentlyContinue

connect-vbrserver -server localhost

#Query to get all jobs in the time period in hours that have processed data (Data ne 0 ) and have completed

$vbrtasksessions = (Get-VBRBackupSession |

Where-Object {($_.EndTime -ge (Get-Date).addhours(-168) -or $_.CreationTime -ge (Get-Date).AddHours(-168) -or $_.State -eq "Working")}) | Get-VBRTaskSession | Where-Object {$_.Status -notmatch "Idle|InProgress|Pending|Fail"-and $_.Info.Progress.TotalSize -ne "0"}

#Get Backup Job and VM

#$getOne = $vbrtasksessions | ? {$_.JobName -eq "%Veeam Backup or Backup Copy Name%" -and $_.Name -eq "%Name of Server%"}

#Or Get Whole Backup Job and average between all the machines

#$getOne = $vbrtasksessions | ? {$_.JobName -eq "%Veeam Backup or Backup Copy Name%"}

#Cannot Get Total usedSize in Backup

Echo ListofAllFullSizesInBytes $getOne.Info.Progress.TotalSize

#Echo AverageTotalFullBackupSizeInGB $([Math]::Round([Decimal]($getOne.Info.Progress.TotalSize | Measure-Object -Average).Average/1GB, 2))

Echo ListofIntcrementalSizeinBytess $getOne.Info.Progress.TransferedSize

#Echo AverageIntSizeinGB $([Math]::Round([Decimal]($getOne.Info.Progress.TransferedSize | Measure-Object -Average).Average/1GB, 2))

Echo RateofChangein% ((($getOne.Info.Progress.TransferedSize | Measure-Object -Average).Average/($getOne.Info.Progress.TotalSize | Measure-Object -Average).Average)*100)

Disconnect-VBRServer | out-null
VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Work out how much Space

https://rps.dewin.me/

work out bandwidth

https://rps.dewin.me/bandwidth/

Refs Calculator

http://dewin.me/refs/

 

 

  1. It is not a good idea to target the primary and backup copy job to the same repository as this negates the 321 rule and if an issue was to occur on that repository the primary and secondary copies of data would be lost.
  2. Instead it would be better to use a primary repository configured with ReFS and then have a secondary location configured with Windows Deduplication (Compression should be off instead of auto).
  3. It is not generally considered a good idea to have deduplication for the primary backup chain as this effects performance and increases RTPOs.
  4. The actual space savings depend on the change rate of the source data and the types of chains that are used on the ReFS repository. Also ReFS savings don’t show when viewed through file explorer. To learn more about checking the files savings please take a look at these links;

https://www.virtualtothecore.com/how-much-space-am-i-saving-thanks-to-refs-blockclone/

http://dewin.me/refs/

  1. Some limitation worth mentioning are Windows deduplication is for 4TB files so larger files will not be completely deduped. Also GFS doesn’t show ReFS savings until the point is synthesized as up to this point it will only shows as an incremental and changed blocked cannot be reused by ReFS pointers.

 

 

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Recently we had jobs for a server showing up the error : NFC storage connection is unavailable

Image result for NFC storage connection is unavailable

Turns out a colleague had added a server to the Veeam application with a tape drive to the unit which had actually added that server as a Veeam Backup Proxy!

Jobs actually default to “Use Any Available Proxy” so the Jobs were using the tape drive server as a proxy for some of the Jobs hence the NFC Failure

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Recently multiple Transform jobs for Veeam on a Synology NAS was maxing out the I/O’s of the NAS , with the errors

Previous full backup chain transformation failed Error: Agent: Failed to process method {Transform.Patch}: An unexpected network error occurred.

applying the below registry keys and spacing out the transform fixed the issue 

 

SessTimeout - Reboot Required

Key: HKLM\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters\
DWORD: SessTimeout
This is a value in seconds. Try a value of 600 decimal (10 minutes).
This increases the amount of time the Windows SMB client will wait for a response from an SMB server before it aborts the connection. The default timeout is one minute.

TcpMaxDataRetransmissions - Reboot Required

Key: HKLM\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters
DWORD: TcpMaxDataRetransmissions
Try a value of 10.
This increases the number of times the Windows TCP implementation will retransmit a data segment before it aborts the connection. The default number of retries is five.
VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

The first delay is a long cycle of checking every file mentioned in the VSS writers metadata. There are 8 million of FILESTREAM blobs on that server. This must be what causes this cycle to be so long for that volume.

 

Please download the bundle here: https://storage.veeam.com/Fix_125973_aebe4c1eb0.zip

To install, do the following on Veeam server and any Guest Interaction proxies that you have:
1. Locate C:\Program Files (x86)\Veeam\Backup Transport\GuestInteraction\VSS\VeeamGuestHelpers
2. Rename VeeamVssSupport2003_X86.dll to VeeamVssSupport2003_X86.dll_old
3. Rename VeeamVssSupport2008R2_X64.dll to VeeamVssSupport2008R2_X64.dll_old
4. Rename VeeamVssSupportXP_X86.dll to VeeamVssSupportXP_X86.dll_old
5. Extract new versions of these 3 DLL files from the archive.

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

A Server 2012 R2 started getting VSS Writer Errors randomly in Veeam backup job. The server is running AADconnect which is set to autoupdate

A restart of the VSS SQL Writer which usually fixes this did not resolve the issue.

A look in the event log shows

A VSS writer has rejected an event with error 0x800423f4, The writer experienced a non-transient error. If the backup process is retried,
the error is likely to reoccur.
. Changes that the writer made to the writer components while handling the event will not be available to the requester. Check the event log for related events from the application hosting the VSS writer.

Operation:
PrepareForSnapshot Event

Context:
Execution Context: Writer
Writer Class Id: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}
Writer Name: SqlServerWriter
Writer Instance Name: SQL Server Code-Named ‘Denali’ CTP2:SQLWriter
Writer Instance ID: {6c73bfe9-f82f-4854-bec4-4382c314a583}
Command Line: “C:\Program Files\Microsoft SQL Server\90\Shared\sqlwriter.exe”
Process ID: 4292

With some

SQLVDI: Loc=SignalAbort. Desc=Client initiates abort. ErrorCode=(0). Process=4292. Thread=3940. Client. Instance=LOCALDB#SHDA41B2. VD=Global\{9C831400-DE4D-4364-BA22-A8299CF545FC}4_SQLVDIMemoryName_0.

Looks like you need to download the SQL Fix from here : https://support.microsoft.com/en-us/help/2983175/cumulative-update-package-2-for-sql-server-2012-sp2

Mirror Here : https://mega.nz/#!3AZ3gK7I!kDPhnAlur4XtslKxGXwmbnLitJiwN9R6rK-z4Rh0N-s

 

**Update , fix does not seem to work

A repair on the SQL database and then run the below script

Looks like this might be an issue with AADcoonect

https://github.com/pariswells/pariswells.com/blob/master/fixaadconnectvsswriter.ps1

 

**Update

 ADSync launches an SQL Server Local DB under it’s own user account
– The User Profile Service thinks ADSync is no longer logged on, and unloads the registry
– SQL Server though still has handles to the registry, but they’re invalid now

Detailed explanation:
https://support.microsoft.com/en-us/help/2287297/a-com-application-may-stop-working-on-windows-server-2008-when-a-user

In short: Computer Configuration->Administrative Templates->System->User Profiles->Do not forcefully unload the user registry at user logoff

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

 Error: Unfreeze error: [Backup job failed.
Cannot create a shadow copy of the volumes containing writer’s data.
A VSS critical writer has failed. Writer name: [SqlServerWriter]. Class ID: [{a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}]. Instance ID: [{60e3e5dd-73fc-46d6-ab6b-6b8723df09e3}]. Writer’s state: [VSS_WS_FAILED_AT_PREPARE_SNAPSHOT]. Error code: [0x800423f4].]

I restarted the SQL Server VSS Writer ( Can be done during production as only used for backups ) and a retry still errored.

In the end I needed to reboot the server

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

Recently an Veeam Backup and Replication Software was upgraded from v8 to 9.5 , to fix an issue with Internal Database

Failed to copy item ‘\\network\Location’ Error: The filename, directory name, or volume label syntax is incorrect

this feature is not supported since version 9, so this might work but the product does not guarrantee this feature would work properly, and unfortunately we do not support features that does not supported by the version you are using.

please see this guide: https://helpcenter.veeam.com/docs/backup/hyperv/file_copy_source.html?ver=95

it says “You cannot use deduplicating storage appliances, shared folder backup repositories and shared folders as a source for the file copy job.”

This is annoying as it did work with Older Versions , in the end , I have to add \\network\Location\Folder ( extra folder on the end ) for the source folder instead of \\network\Location

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

A veeam backup showed a warning with the following error

Unable to truncate Microsoft SQL Server transaction logs. Details: RPC function call failed. Function name: [BlobCall]. Target machine: []. RPC error:The remote procedure call failed. Code: 1726

Leaving this to run the following day proved a success so must of been SQL Issue

VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)

For some reason I couldn’t delete credentials from Veeam due to them being used on a Shared Folder that did not existing in a backup repository or Tape to Folder. In the end I had to use SQL remove this.

  1. Verify the record to be deleted by running the following query against the configuration database (VeeamBackup by default);

    SELECT * FROM [backup.model.mrulist]
  2. Run the following query, changing <share path> to match the record to be deleted.

    DELETE FROM [backup.model.mrulist] WHERE url = ‘<share path>’
VN:F [1.9.22_1171]
Rating: 0.0/10 (0 votes cast)
VN:F [1.9.22_1171]
Rating: 0 (from 0 votes)