wiki/infrastructure.md

title	description	published	date	tags
Infrastructure	The hardware and software powering it all	true	2019-06-29T19:10:36.767Z

Basic overview

• Entrypoint to network is a Linksys WRT1200AC
• Main HP DL380 G6 server
• File server is a Custom Intel SC5000 BASE

Router

Self assigned WAN IP address, easily probed via our true domain fosc.etsit.upct.es

Running latest Openwrt trunk, updated every handful of weeks

Performance can be overviewed at: router.fosc.space

Main Server

Its hostname is just fosc

Runs NixOS with some nice configs and acts as a docker server. As usual for NixOS, it is configured from .nix files in /etc/nixos

These configs are not intended to be touched by mortals, but can be seen by members so they can help in home server setups

An extensive docker-compose setup is present at /var/docker/ Go there if you have been given access to deploy a stack

RAID setup

NixOS boots from an independent f2fs 250GB SSD.

Real storage is available in a ZFS "RAID10" composed of the 4 mechanical disks. Due to RAID card annoyances, replacing a disk requires a reboot (and probably more intervention)

  pool: tank
 state: ONLINE
  scan: scrub repaired 0B in 0 days 00:10:44 with 0 errors on Mon Jun  3 13:53:46 2019
config:

	NAME                                        STATE     READ WRITE CKSUM
	tank                                        ONLINE       0     0     0
	  mirror-0                                  ONLINE       0     0     0
	    scsi-3600508b1001c2d61e4e90873322fc373  ONLINE       0     0     0
	    scsi-3600508b1001cbb53e902f820f105614f  ONLINE       0     0     0
	  mirror-1                                  ONLINE       0     0     0
	    scsi-3600508b1001cdacdaa62eca0df4ed607  ONLINE       0     0     0
	    scsi-3600508b1001cfd528529622e93c7f74f  ONLINE       0     0     0

errors: No known data errors

Traefik

SSL terminator / HTTP proxy for the network, hogs port 80 and 443 (among others) and will fetch different webpages based on domain

All HTTP(S) services must be accessed over this proxy

Long term storage

Files served by the web servers and backups are stored at /mnt/gordo

Backups

Backups occur weekly at 04:00 using the restic software Encryption key has been shared among the head members

File Server

This server is currently out of comision. Backups have been taken to offline disks. {.is-danger}

As you would have expected, gordo

Also runs NixOS, using a simpler configuration than the main server.

Its primary function is running an NFS server and beinbg a torrent seedbox. It also has a dormant traefik+docker setup, ready to replace the main server in case of emergency.

RAID setup

Three 1TB SATA disks in raidz (RAID5). The first disk seen by the BIOS contains the GRUB bootloader and a small boot partition aside from one of the three ZFS partitions

Only one disk can fail at the same time. Be careful out there

  pool: tank
 state: ONLINE
  scan: scrub in progress since Wed Jun 12 16:44:20 2019
        584G scanned at 406M/s, 209G issued at 146M/s, 959G total
        0B repaired, 21.84% done, 0 days 01:27:43 to go
config:

        NAME                                 STATE     READ WRITE CKSUM
        tank                                 ONLINE       0     0     0
          raidz1-0                           ONLINE       0     0     0
            ata-ST31000528AS_9VP0TRE9        ONLINE       0     0     0
            ata-ST31000528AS_9VP0TRYB        ONLINE       0     0     0
            ata-ST31000528AS_9VP0TS1L-part3  ONLINE       0     0     0

errors: No known data errors