Using Restic for Backups on Windows

I recently went about setting up a home server to replace a failed NAS appliance. This time, instead of proprietary software, I used Debian Linux and configured everything myself. Because it is storing a bunch of important files, an automated online backup system was an important part of the new build.

I had previously used Arq for backing up our home network. It is awfully powerful and easy to use. I especially appreciate that it lets you run backups to numerous cloud providers. Unfortunately, it only runs on Windows and Mac. I could keep using it by letting my main Windows computer back up the contents (by reading from a shared drive), but something running on the Linux server itself would be much better.

After a round of searching, I settled on Restic. Restic is a lot like Arq, except that it runs on Windows, Mac, and Linux. It’s also free and open source. Unfortunately, it is command-line only, so it takes a bit more work to set it up.

The Linux setup was easy, but it took a couple of tricks to get it working the way I wanted on Windows…

Setup on Linux

There is lots of good documentation for setting up Restic backups on Linux. This guide also shows setting up an S3 bucket in AWS: https://restic.readthedocs.io/en/stable/080_examples.html#setting-up-restic-with-amazon-s3

Because I use Debian for my home server, I used systemd Services and Timers to run the backups on a schedule. It was easy to get going, and works great. I highly suggest using systemd-creds to keep your repository key encrypted in your systemd service files.

You should also make sure to run this command after installing Restic: sudo restic self-update. One of the weaknesses (and strengths) of Debian is that the package feeds are slow to take feature updates. The version apt installed still worked, but it was pretty old, and there were some important updates in that time.

Installing on Windows

Setting it up on Windows isn’t very different from Linux. It might be intimidating for an average Windows user, but a power user should have no trouble. winget makes it particularly easy:

winget install --exact --id restic.restic --scope Machine --force

Configuring Regular Backups on Windows

I spent a bit of time experimenting and iterating before I settled on the following configuration. It’s not a requirement for using Restic, but I like the way it works.

First of all, I like to keep the configuration and cache files in ProgramData. You can use these commands to create the directories required for the scripts below:

# Create the directory structure:
New-Item -ItemType Directory C:\ProgramData\ResticBackup\Settings -Force
New-Item -ItemType Directory C:\ProgramData\ResticBackup\Cache -Force
New-Item -ItemType Directory C:\ProgramData\ResticBackup\Logs -Force

I also use text files for listing the includes and excludes. This way I can easily look the list up or adjust it without touching the scheduled task or PowerShell code.

"C:\Users\$Env:USERNAME
C:\ProgramData\ResticBackup\Settings
" | Out-File C:\ProgramData\ResticBackup\Settings\Includes.txt

"# Code
node_modules
C:\Code\**\bin
C:\Code\**\obj
C:\Code\**\build
.vscode
.nuget\packages
.idea\libraries

# User directories
C:\Users\**\Temp
C:\Users\**\Google\Chrome
C:\Users\**\INetCache
C:\Users\**\MicrosoftEdge\Cache
C:\Users\**\Mozilla\Firefox\**\cache*
" | Out-File C:\ProgramData\ResticBackup\Settings\Excludes.txt

This is the PowerShell script that runs the backup. Write the contents to C:\ProgramData\ResticBackup\Settings\Invoke-ResticBackup.ps1:

#Requires -Version 7
param
(
    [Switch]
    $WhatIf
)

Set-StrictMode -Version 3.0
$ErrorActionPreference = 'Stop'

$LogPath = "C:\ProgramData\ResticBackup\Logs"
$LogFilePath = "$LogPath\$(Get-Date -Format 'yyyy-MM-dd_HH-mm-ss')_Backup.txt"
Start-Transcript -Path "$logFilePath" -UseMinimalHeader

$ResticRepositoryKeyFile = "C:\ProgramData\ResticBackup\Settings\ResticRepositoryKey.$Env:USERNAME.txt"
if (-not (Test-Path $ResticRepositoryKeyFile))
{
    throw "repository key file not found for user: $ResticRepositoryKeyFile"
}

$backupParams =
    $(
        '--repo', 's3:s3.amazonaws.com/<your-bucket-name>',
        '-o', 's3.region=<your-region>',
        '-o', 's3.storage-class=ONEZONE_IA'
    )

$Env:AWS_PROFILE = 'restic'
$Env:AWS_SHARED_CREDENTIALS_FILE = "C:\Users\$Env:USERNAME\.aws\credentials"

if ($WhatIf)
{
    $backupParams += '--dry-run'
}

$Env:RESTIC_PASSWORD_COMMAND = "pwsh -NoProfile -NonInteractive -Command '(Get-Content `"C:\ProgramData\ResticBackup\Settings\ResticRepositoryKey.$Env:USERNAME.txt`" -Raw).Trim() | ConvertTo-SecureString | ConvertFrom-SecureString -AsPlainText'"
$Env:RESTIC_PROGRESS_FPS = '0.016666'

restic backup @backupParams `
    --files-from "C:\ProgramData\ResticBackup\Settings\Includes.txt" `
    --iexclude-file "C:\ProgramData\ResticBackup\Settings\Excludes.txt" `
    --cache-dir 'C:\ProgramData\ResticBackup\Cache' `
    --use-fs-snapshot `
    --no-scan `
    --verbose `
    2>&1 | Out-String -Stream | Tee-Object "$LogPath\restic-progress.txt"

You can use this command to schedule the task (running as user SYSTEM, twice daily):

$action = New-ScheduledTaskAction -Execute 'pwsh.exe' `
    -Argument "-NonInteractive -NoProfile -File `"C:\ProgramData\ResticBackup\Settings\Invoke-ResticBackup.ps1`""

$trigger1 = New-ScheduledTaskTrigger -Daily -At 7:10am
$trigger2 = New-ScheduledTaskTrigger -Daily -At 7:10pm
$settings = New-ScheduledTaskSettingsSet -ThrottleLimit 1 -MultipleInstances IgnoreNew -DontStopIfGoingOnBatteries
$principal = New-ScheduledTaskPrincipal -UserId "SYSTEM" -LogonType ServiceAccount -RunLevel Highest

$task = New-ScheduledTask -Action $action -Trigger $trigger1,$trigger2 -Settings $settings -Principal $principal

Register-ScheduledTask -TaskPath 'Backup' -TaskName 'Restic S3 Backup' -InputObject $task -Force

Storing Passwords Securely on Windows

If you were paying close attention above, you may have noticed the username variable in the repository key file. That’s because the key is stored in an encrypted format (via ConvertFrom-SecureString), and the encryption key it uses is different for each user of the computer.

ConvertFrom-SecureString has a -Key parameter so that you can use the same encrypted string for multiple users. The problem with this approach is that you then have to protect the key used to encrypt the secret, and end up with the same problem again.

This mechanism is a little annoying, but it keeps the key safe. It would be a lot simpler to store your unencrypted repository key in a text file and let Restic use this instead. I don’t recommend this, however, because anyone who can access that file could then read your backups.

I use this script to write my own copy of the encrypted repository key:

$ResticRepositoryKey = Read-Host -AsSecureString -Prompt 'Restic Repository Key'
$ResticRepositoryKey | ConvertFrom-SecureString `
    | Out-File "C:\ProgramData\ResticBackup\Settings\ResticRepositoryKey.$Env:USERNAME.txt"

Saving the key so that the system user can decrypt it is a little more challenging. There is no way (that I’ve found) to encrypt a secret for another user with ConvertFrom-SecureString, so you have to run the code that encrypts it as that user. Because you can’t log in as SYSTEM directly, I use a scheduled task. The running task needs access to the key, however, so I temporarily write the unencrypted key to a file.

Write the following to Save-EncryptedResticRepositoryKeyForCurrentUser.ps1:

#Requires -Version 7
Set-StrictMode -Version 3.0
$ErrorActionPreference = 'Stop'

$LogPath = 'C:\ProgramData\ResticBackup\Logs'
$LogFilePath = "$LogPath\$(Get-Date -Format 'yyyy-MM-dd_HH-mm-ss')_SaveEncryptedKey_$Env:USERNAME.txt"
Start-Transcript -Path "$logFilePath" -UseMinimalHeader

$ResticRepositoryKey = (Get-Content 'C:\ProgramData\ResticBackup\Settings\UnencryptedRepositoryKey.txt' -Raw).Trim() `
    | ConvertTo-SecureString -AsPlainText

$ResticRepositoryKey | ConvertFrom-SecureString `
    | Out-File "C:\ProgramData\ResticBackup\Settings\ResticRepositoryKey.$Env:USERNAME.txt"

This script writes the unencrypted repository key to a file, then configures a scheduled task to encrypt it as the SYSTEM user. It then runs the scheduled task, waits a few seconds for it to finish, and (crucially) deletes the unencrypted key file.

$ResticRepositoryKey = Read-Host -AsSecureString -Prompt 'Restic Repository Key'
$ResticRepositoryKey | ConvertFrom-SecureString -AsPlainText `
    | Out-File 'C:\ProgramData\ResticBackup\Settings\UnencryptedRepositoryKey.txt'

$scriptPath = "C:\ProgramData\ResticBackup\Settings\Save-EncryptedResticRepositoryKeyForCurrentUser.ps1"
$action = New-ScheduledTaskAction -Execute 'pwsh.exe' `
    -Argument "-NonInteractive -NoProfile -File `"$scriptPath`""
$settings = New-ScheduledTaskSettingsSet -ThrottleLimit 1 -MultipleInstances IgnoreNew -DontStopIfGoingOnBatteries
$principal = New-ScheduledTaskPrincipal -UserId "SYSTEM" -LogonType ServiceAccount -RunLevel Highest
$task = New-ScheduledTask -Action $action -Settings $settings -Principal $principal
Register-ScheduledTask -TaskPath 'Backup' -TaskName 'Encrypt Restic Repository Key for SYSTEM User' -InputObject $task -Force

Start-ScheduledTask -TaskPath 'Backup' -TaskName 'Encrypt Restic Repository Key for SYSTEM User'

Start-Sleep -Seconds 5
Remove-Item 'C:\ProgramData\ResticBackup\Settings\UnencryptedRepositoryKey.txt'

Running Restic Manually

If you want to run any manual Restic commands (viewing backups, retrieving files, pruning history, etc.), and you don’t feel like pasting your repository key every time, you can use this helpful one-liner:

$Env:RESTIC_PASSWORD_COMMAND = "pwsh -NoProfile -NonInteractive -Command '(Get-Content `"C:\ProgramData\ResticBackup\Settings\ResticRepositoryKey.`$Env:USERNAME.txt`" -Raw).Trim() | ConvertTo-SecureString | ConvertFrom-SecureString -AsPlainText'"

It allows Restic to decrypt and use your encrypted repository key every time it needs it. This is the same mechanism used in the backup script.

Regarding S3 credentials

I am storing my backups in AWS S3, and using the AWS CLI’s own configuration mechanism to store the AWS API key for me. It doesn’t seem to be encrypted, which is not my favourite, but I’ve decided that this is okay for me. The key is associated with a user that has only the permissions needed to write backups into the one S3 bucket I use for them.

If you want to be extra cautious, you could use a trick like I use for the repository key.

Other Considerations

For a fully-functional, self-sustaining backup system, there are a few more things you need to set up:

• regular pruning of old/unnecessary data
• some kind of monitoring to tell you if the backups stop running
• a routine/process to periodically check that you can actually pull files from your backup
• if the backup destination supports it, some mechanism to lock/version files for a set amount of time will ensure that your backups can’t be destroyed secretly (a common trick used in ransomware attacks)

Would anyone be interested in me writing about these things? Comment below with any particular questions you have.

Results

I’ve been using this to back up my personal laptop for over a year now, and it’s been working great. It’s fast and effective. It works even when I’m travelling. And I don’t notice a performance hit if it runs while I’m working.

Arq was good to me, but I’m glad I switched.

See also

• Making a Health Tracker With GitHub Spark
• Copying Files to AWS ECS Fargate Instance via Base64
• Connect to Remote Desktop from Outside Your Home Securely with SSH
• Disambiguating Scalability
• My Computer Setup