README.md (7359B)
1 The scripts in this folder are used by a jenkins slave to build and test 2 neptools. To use it you'll need an amd64 sysroot, a qemu image of win 7 (or 3 later) with ssh, wine, msvc includes+libs, patched clang, gcc, and probably 4 else. Documetation is mostly non-existing. 5 6 Sysroot creation 7 ================ 8 9 You need docker to run this script, it will place the base sysroot in your 10 working directory. Readline and its deps (ncurses, tinfo) were required by ljx 11 cli, TODO remove them. 12 13 ```sh 14 docker run --rm jimbly/steamrt-amd64-gcc bash -c 'sed -i -re "s/([a-z]{2}\.)?archive.ubuntu.com|security.ubuntu.com/old-releases.ubuntu.com/g" /etc/apt/sources.list && apt-get update >&2 && apt-get -y install libreadline6-dev >&2 && dpkg-query -L libc6 libc6-dev linux-libc-dev libgcc1 gcc-4.6 libreadline6-dev libncurses5-dev libtinfo-dev libstdc++6-4.6-dev | grep -E "^(/usr/include/|/usr/lib/|/lib/)" | xargs tar cvh --no-recursion' | tar x 15 rm usr/lib/x86_64-linux-gnu/lib{readline,ncurses,tinfo}.so 16 mv usr/lib/x86_64-linux-gnu/lib{*_nonshared,readline,ncurses,tinfo}.a ./ 17 rm usr/lib/x86_64-linux-gnu/*.a 18 rm -r usr/lib/x86_64-linux-gnu/{gconv,libc} 19 rm usr/lib/gcc/x86_64-linux-gnu/*/{lto1,lto-wrapper} 20 mv lib{*_nonshared,readline,ncurses,tinfo}.a usr/lib/x86_64-linux-gnu/ 21 mkdir lib64 22 ln -s ../lib/x86_64-linux-gnu/ld-2.15.so lib64/ld-linux-x86-64.so.2 23 ``` 24 25 Windows VM creationg 26 ==================== 27 28 There are two windows VMs at the moment, a 32-bit XP SP3 with zero updates and a 29 64-bit Win7 SP1 + probably some updates. I didn't take any notes when I made the 30 win7 VM, but with XP it went something like this: 31 32 * Qemu settings: `-enable-kvm -nodefaults -machine pc-i440fx-2.7 -m 1024 33 -cpu pentium2,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff 34 -smp cores=2,sockets=1 -rtc base=localtime 35 -drive file=$image_file,id=disk,format=qcow2,discard=unmap,detect-zeroes=unmap,cache=unsafe,if=none 36 -device virtio-blk,drive=disk 37 -netdev user,restrict=on,ipv6=off,id=mynet,hostfwd=tcp:127.0.0.1:2222-:22 38 -device virtio-net,netdev=mynet,id=mynic -vga std -usbdevice tablet` 39 XP itself can run with pentium, but ucrt used by vc 14.11 requires pentium2. 40 Also, with kvm it's not possible to disable every feature not supported by 41 pentium(2) but supported by your host, you need to use CPU emulation if you 42 need that, but it's very slow. My recommendation is to install the OS with 43 KVM, then disable and create a snapshot after it booted. 44 * Get an original WinXP SP3 iso 45 * Install with ~default settings, user password not needed 46 * Essential settings: set visual settings to performance, remove unneeded 47 windows components, disable windows firewall (and probably the whole security 48 center nonsense) 49 * Computer -> Manage -> Services, and disable "Error Reporting Service" 50 (otherwise if the test crashes, it'll hang the SSH connection since it'll wait 51 for the non-existing user to close the graphical dialog that is being 52 redirected to /dev/null) 53 * Install MSVC redist 12 (2013) and 14.11 (2017) 54 * SSH server: [freesshd] Manually add a user with some dummy password. As of 55 OpenSSH-8.6, you'll have to specify to `-o HostKeyAlgorithms=ssh-rsa,ssh-dss` 56 to be able to connect to the server. If you want to run "shell" commands, you 57 neet to run it as `ssh user@host cmd /c 'your commandline'`. 58 59 Alternative SSH servers: 60 * [kpym]: can't run commands, only supports interactive sessions, garbage. 61 * [bitvise ssh][bitvise]: intaller crashed with an unknown instruction when I 62 tried. It needs more than a pentium cpu, but it doesn't document which. 63 * [sshwindows]: didn't try, last release in 2004 64 * Microsoft's OpenSSH: "official" port of OpenSSH to windows. Requires win7 (and 65 a fair amount of swearing, as the provided PowerShell installer doesn't 66 fucking work on win7. Also have fun finding the installer online, all they 67 document is how to install this shit on botnet 10). Requires you to have a 68 password on your windows user if you want to login with password. Used in the 69 win7 vm, but if I reinstall it, I'm not sure I'll use this again. 70 71 [freesshd]: http://www.freesshd.com/?ctt=download 72 [kpym]: http://www.kpym.com/2/kpym/download.htm 73 [bitvise]: https://www.bitvise.com/ 74 [sshwindows]: http://sshwindows.sourceforge.net/ 75 76 General tips: 77 * You basically have two options to authenticate with an SSH server: password 78 and public key based. Public key based is the normally recommended way to use 79 in scripts, but it's a pain in the ass to set up (you have to copy the pubkey 80 of your build chroot's BEFORE you make the final disk image and copy it to the 81 build machine, you can't easily use the VM from other machines unless you copy 82 *every* relevant pubkey there. It's much easier to get `sshpass` and just pass 83 the password on the command line (and use `-o StrictHostKeyChecking=no` to get 84 rid of the host key verification prompt). It's pointless to argue about 85 security when your user has RW access to the SSH server's disk image and the 86 SSH server is never exposed to a network anyway. 87 * OpenSSH has a fantastic security anti-feature that it automatically writes the 88 pubkey of every server you ever connect to, then it complains that the key 89 changed after you try to connect to a different VM, forcing you to manually 90 edit the known hosts files, or more likely just `rm ~/.ssh/known_hosts` 91 because ain't nobody got time for that shit. The CI scripts solves this 92 problem by running SSH in a mount namespace where it doesn't have write access 93 to `~/.ssh`... 94 95 How to create an image that's small and can be loaded fast: 96 1. Install & configure everything you need, copy [sdelete] to the VM 97 2. It's a good idea to make a snapshot with `qemu-img snapshot -c $snapshot_name 98 $disk_image` if you use a qcow2 disk image. 99 3. Unless you want to inflate your disk image to its maximum size, make sure you 100 use `discard=unmap,detect-zeroes=unmap` with your disk drive. 101 4. Boot windows, run `sdelete -z c`, (delete sdelete), shutdown 102 5. Now start qemu with `-snapshot`, wait until windows boots and network/ssh is 103 ready 104 6. Qemu console: `migrate -i "exec:cat > $state_file"` then quit 105 7. You can try it with `-snapshot` and `-incoming "exec:cat $state_file"` 106 8. If it's working, time to shrink these huge images. For the base disk image: 107 `qemu-img convert -p -f qcow2 -O qcow2 -o compression_type=zstd -c $disk_img 108 $output_img`. For the migration state file, just compress it with whatever 109 you like, I used `zstd --ultra -22`. Note that qemu compresses with the 110 default 3 compression level, if you want your disk images to be a few percent 111 smaller while spending 5x more time compressing it, apply 112 `qemu-6.0.0-zstd.patch` and recompile qemu. 113 9. Test that it still works. Run qemu with `$output_img` as disk image, still 114 with `-snapshot` and `-incoming "exec:zstdcat $compressed_state_file"`. 115 10. If you fucked up anything, go back to the snapshot you made with `qemu-img 116 snapshot -a $snapshot_name $disk_image`, fix the problems, then try again 117 from step 2. 118 119 If you use an OS with discard/trim support it might be better to use 120 virtio-scsi-pci/scsi-hd instead of virtio-blk, and issue an `fstrim` (or 121 whatever command is required for your OS) instead of manually overwriting every 122 unused sector with zeroes. 123 124 [sdelete]: https://docs.microsoft.com/en-us/sysinternals/downloads/sdelete