Transcription
Building a security appliancebased on FreeBSDBSDCanOttawa 20191
Mariusz ttawa 20192
3
Dataencryption4
DataencryptionStorage5
ExternalstorageDataencryptionStorage6
RemoteaccessExternalstorageDataencryptionStorage7
yptionStorage8
Data Encryption9
Data Encryption GBDE GELI native ZFS encryption10
GBDE - Geom Based Disk Encryption FreeBSD 5.0 AES-CBC 128bits Different key for each write CPU overhead disk space overhead11
GELI Many cryptographic algorithms AES-XTS AES-CBC Blowfish-CBC Camellia-CBC 3DES-CBC Integrity verification (HMAC) Don’t have such overheads like GDBE One-time key12
Keeping encryption keyAppliance: Use memstick Need only during boot Initialize during first bootVM: Use passphrase Use no encryption13
Storage14
Storage ZFS UFS15
ZFS checksums snapshots compression RAIDZ16
ZFS - checksum fletcher2if (id 0 id channels alloc)if (id 0 id channels alloc) fletcher4jle 30jl 3039 45 08 7c 1a 8b 4539 45 08 7e 1a 8b 450111110001111110 sha256 sha512 skein17
ZFS - compression GZIP lz4 ZSTD# zfs list -o r/crashlz411.17x18
ZFS - compression GZIP lz4 ZSTD# zfs list -o r/crashlz411.17xProblem: What if customer want to backup the data?19
ZFS - snapshotsA snapshot is a read-only copy of a file system or volume.Snapshots can be created almost instantly, and they initiallyconsume o additional disk space within the pool. However, as datawithin the active dataset changes, the snapshot consumes diskspace by continuing to reference the old data, thus preventing thedisk space from being freed.https://docs.oracle.com/cd/E23824 01/html/821-1448/gbciq.html20
Snapshots - cluster multi-masterMaster0New DatacontinuousreplicationMaster1New Data21
Snapshots - cluster multi-masterMaster0New DatacontinuousreplicationMaster1llatosseccaatadNew Data22
Snapshots - cluster multi-master# zfs a/data/12345678/dumps/data/local/dumps23
Snapshots - cluster multi-master# zfs list -t snapshotNAMEUSEDAVAILREFER data/data/local/dumps@201801300520380-192k-24
ZFS sending & receiving snapshotsBefore r317414:After r317414: ZFS decompress FS to send ZFS FS send over SSH manual compress FS to reduce ZFS FS receive over SSHlatency sending over SSH manual decompress FS receivedover SSH ZFS compress FS which wasreceived25
Downside of using ZFS snapshots Data loss after rollback Can’t rollback ZFS changes Snapshots can take a lot of space on clustermulti-master26
Downside of using ZFS snapshots Data loss after rollback stnioCan’t rollbackZFS changespkcehcs Snapshots can takekr a lot of space on clusteramkomulti-masterob27
Downsides of ZFS Not enough RAM to import pool No full disk encryption If something will go very wrong we still want tobe able to do something What about factory reset?28
Read only file system - UFS GELI&ZFS for customer data Contains read-only operating system Data are not encrypted If something goes wrong we can still boot from it Try to reflect some ZFS promises29
Read only file system - UFS# gpart show -l ada0 stem0-0[bootme] 408116793067276data0(5.4T)(64K)30
RAIDZ231
Reflect RAIDZ2 with UFSNamemirror/system0StatusComponentsCOMPLETE gpt/system0-0 (ACTIVE)gpt/system0-1 (ACTIVE)gpt/system0-2 (ACTIVE)gpt/system0-3 (ACTIVE)gpt/system0-4 (ACTIVE)gpt/system0-5 (ACTIVE)32
Reflect RAIDZ2 with SWAPNamemirror/swap0StatusComponentsCOMPLETE gpt/swap1 (ACTIVE)gpt/swap2 (ACTIVE)gpt/swap0 (ACTIVE)mirror/swap1COMPLETE gpt/swap3 (ACTIVE)gpt/swap4 (ACTIVE)gpt/swap5 (ACTIVE)33
Upgrade steps34
Upgrade steps - Boot from system0# gpart show -l ada0 4011721045101ada04012816883886082system0-0[bootme] 408116793067276data0(5.4T)1GPTboot0(5.5T)(64K)35
Upgrade steps - override system1 and set bootonce# gpart show -l ada0 4011721045101ada04012816883886082system0-0[bootme] (4.0G)838877683886083system1-0[bootonce, bootme] boot0(5.5T)(64K)36
Upgrade steps - reboot37
Upgrade steps - bootloader removes bootme# gpart show -l ada0 stem0-0[bootme] (4.0G)838877683886083system1-0[bootonce] )38
Upgrade steps Create zfs snapshot Upgrade error accrued Reboot machine39
Upgrade steps - boot from partition with bootme# gpart show -l ada0 stem0-0[bootme] (4.0G)838877683886083system1-0[bootonce] )40
Upgrade steps - rollback# zfs rollback -R data@upgrade# gpart show -l ada0 stem0-0[bootme] (4.0G)838877683886083system1-0[bootfailed] )41
Upgrade steps - upgrade succeeded# gpart show -l ada0 stem0-0[bootme] (4.0G)838877683886083system1-0[bootonce] )42
Upgrade steps - upgrade succeeded# gpart show -l ada0 stem0-0(4.0G)838877683886083system1-0[bootme] )43
Hot plugnotify 20 {match "system" "DEVFS";match "type" "CREATE";match "cdev" " ada[0-9] ";action "/usr/local/bin/newdisk cdev";};44
External Storage45
External storage NFS iscsi SAN over FC46
External storage - NFS NFS No encryption No authorization iscsi Is it corporate solution? Able to mount on multiple machines SAN over FC47
External storage - iscsi NFS iscsi SAN over FC Encryption Authorization Is it corporate solution? Not able to mount on multiplemachines48
External storage - SAN over FC NFS iscsi SAN over FC Encryption Authorization It is a corporate solution Not able to mount on multiplemachines49
External storage - SAN over FC NFS iscsi SAN over FC with GELI Encryption Authorization It is a corporate solution Not able to mount on multiplemachines50
Redundancy Use at least two FC cards Combine multiple luns with gmultipath51
Remote access52
Access the box Through SSH We don’t want to customize our builds perclient In theory we can have an key per client SSH keys Hard to hijack What if your engineer change the job? We have to be in customer network53
Access the box - exotic IPMI Some video conference (like webex) No SSH keys So maybe password after all? But password is easy to hijack What if yours enginner change the job?54
Implementing S/Key (whlkey)55
Implementing S/Key (whlkey) We configure it as: 50 keys per day The key length is 16 chars Key is rotated every day Unified password: O 0, I l, etc. The secret is stored in some trusted machine The engineer can only get keys for this week56
Process security57
Basic problem You can’t build everything from scratch You can’t audit everything Who you really trust?58
Basic problem You can’t build everything from scratch You can’t audit everything Who you really trust?Security stops where the trustbegins59
Privileged separationPrivilegedSIMPLE IPCUnprivileged Reduce TCB Simple communication60
Privileged process Have access to: DB Storage Network Authenticate unprivileged process Extend capabilities of unprivileged process61
Unprivileged process Have access to storage by single FD Have access to network by single/two FD Implements complicate logic Is sending a simple commands asking privileged process Limited RAM Limited CPU time62
Privileged separation with FreeBSDCapcisumPrivilegedlibnvUnprivileged63
Capsicum tight sandboxing (cap enter(2)) capability rights (cap rights limit(2))64
Libnv nvlist create Types: nvlist add {type} string nvlist get {type} number nvlist take {type} bool nvlist move {type} nvlist nvlist send descriptor nvlist recv binary nvlist destoy array65
Privileged separation - is it PFreeTDSfreetypelibX1166
Privileged separation - network daemon Privileged process iswaiting for connection67
Privileged separation - network daemon Privileged process isClientwaiting for connection New connection fromclient68
Privileged separation - network daemon Privileged process isClientwaiting for connection New connection fromclient Fork and createunprivileged process69
Privileged separation - network daemon Privileged process isClientwaiting for connection New connection fromcredentialscredentialsclient Fork and createunprivileged process Client is authenticating70
Privileged separation - network daemon New connection fromclientClient Fork and createunprivileged processRamCPU Client is authenticating Privilegiat process israising unprivilegedprocess limits71
Privileged separation - network daemon Fork and createClientunprivileged process Client is authenticating Privilegiat process israising unprivilegedprocess limits Creating connection tothe serverServer72
Privileged separation - network daemon Client is authenticatingClient Privilegiat process israising unprivilegedprocess limits Creating connection tothe server Pass connection tounprivileged processServer73
Privileged separation - network daemon Client is authenticatingClient Privilegiat process israising unprivilegedprocess limits Creating connection tothe server Pass connection tounprivileged processServer74
Privileged separation - network daemon Privilegiat process isClientraising unprivilegedprocess limits Creating connection tothe server Pass connection tounprivileged process Create a dump fileDump fileServer75
Privileged separation - network daemon Privilegiat process isClientraising unprivilegedprocess limits Creating connection tothe server Pass connection tounprivileged process Create a dump fileDump fileServer Pass dump file76
Privileged separation - network daemon Privilegiat process israising unprivilegedClientprocess limits Creating connection tothe serverDump file Pass connection tounprivileged process Create a dump fileServer Pass dump file77
Other methods Jails CloudABI78
Thank you!Mariusz ttawa 201979
Building a security appliance based on FreeBSD BSDCan Ottawa 2019 1. Mariusz Zaborski 2 m.zaborski@fudosecurity.com oshogbo@FreeBSD.org https://oshogbo.vexillium.org @oshogbovx BSDCan Ottawa 2019. 3. 4 Data encryption. 5 Data encryption Storage. 6 Data encryption Storage External storage. 7 Data encryption Storage External storage Remote access .
