diff --git a/_toc.yml b/_toc.yml index 49af0c44..4ae477bb 100644 --- a/_toc.yml +++ b/_toc.yml @@ -23,7 +23,7 @@ parts: - file: infrastructure-guide/infrastructure_systems sections: - file: infrastructure-guide/pxeserv01 - - file: infrastructure-guide/boot00 + - file: infrastructure-guide/sysdb_server - file: infrastructure-guide/puppet01 - file: infrastructure-guide/repo00 - file: infrastructure-guide/repo01 diff --git a/admin-guide/architecture/networking.rst b/admin-guide/architecture/networking.rst index faf454ee..2823611e 100644 --- a/admin-guide/architecture/networking.rst +++ b/admin-guide/architecture/networking.rst @@ -62,7 +62,7 @@ In addition, the following: ======= ============ ================ =================== Source Destination Ports Purpose ------- ------------ ---------------- ------------------- -any boot00 UDP/69, 80, 443 PXE/Kickstart +any ??? UDP/69, 80, 443 PXE/Kickstart ======= ============ ================ =================== Finally, having DHCP is helpful, but not necessary. diff --git a/admin-guide/deployment/_static/legacy_boot.puml b/admin-guide/deployment/_static/legacy_boot.puml index 9a73b79d..2d5baea9 100644 --- a/admin-guide/deployment/_static/legacy_boot.puml +++ b/admin-guide/deployment/_static/legacy_boot.puml @@ -4,7 +4,7 @@ actor admin participant "BIOS Node\n(legacy boot)" as bootnode participant "QIP DHCP" as dhcp participant "pxeserv01.psi.ch" as pxeserver -participant "boot00.psi.ch" as sysdb +participant "sysdb.psi.ch" as sysdb participant "repo00.psi.ch" as reposerver admin -> bootnode : boots up @@ -29,7 +29,7 @@ note right to "lxprod". At some places this is done the opposite by linking hostname to MAC to "lxprod". Having the **link to "lxprod" loads directly** - **the iPXE kernel "boot00.lkrn"**, shortcutting + **the iPXE kernel "ipxe.lkrn"**, shortcutting the manual selection below. end note bootnode -> pxeserver : load PXE config tftp:/tftpboot/pxelinux.cfg/$HEX-IP @@ -44,7 +44,7 @@ note right comments from 2013. end note admin -> bootnode : selects "Red Hat 7 Network Install" -bootnode -> pxeserver : load iPXE kernel tftp:/tftpboot/boot00.lkrn +bootnode -> pxeserver : load iPXE kernel tftp:/tftpboot/ipxe.lkrn note right https://git.psi.ch/linux-infra/ipxe-build The following steps are by embed/pxe.ipxe diff --git a/admin-guide/deployment/_static/uefi_boot.puml b/admin-guide/deployment/_static/uefi_boot.puml index cc1deb4e..221d7630 100644 --- a/admin-guide/deployment/_static/uefi_boot.puml +++ b/admin-guide/deployment/_static/uefi_boot.puml @@ -4,7 +4,7 @@ actor admin participant "UEFI Node" as bootnode participant "QIP DHCP" as dhcp participant "pxeserv01.psi.ch" as pxeserver -participant "boot00.psi.ch" as sysdb +participant "sysdb.psi.ch" as sysdb participant "repo01.psi.ch" as reposerver admin -> bootnode : boots up diff --git a/admin-guide/deployment/infrastructure.rst b/admin-guide/deployment/infrastructure.rst index dc498437..37c030eb 100644 --- a/admin-guide/deployment/infrastructure.rst +++ b/admin-guide/deployment/infrastructure.rst @@ -17,7 +17,7 @@ other for production, with the following hostnames: +--------------------+----------------+-----------------+ | Role | testing host | production host | +====================+================+=================+ -| sysdb server | boot00-test | boot00 | +| sysdb server | - | sysdb.psi.ch | +--------------------+----------------+-----------------+ | puppet server | puppet00-test | puppet01 | +--------------------+----------------+-----------------+ diff --git a/admin-guide/deployment/ipxe.rst b/admin-guide/deployment/ipxe.rst index 93075401..8de974e1 100644 --- a/admin-guide/deployment/ipxe.rst +++ b/admin-guide/deployment/ipxe.rst @@ -12,7 +12,7 @@ Process When a system PXE boots, the DHCP server will boot either pxelinux in legacy mode or grub on UEFI. It is still possible to load ipxe from either of these. -With the right option, the boot process is pointed to ``boot00.psi.ch`` and one +With the right option, the boot process is pointed to ``sysdb.psi.ch`` and one of the following boot images: - ``ipxe.efi`` for systems using UEFI @@ -21,12 +21,12 @@ of the following boot images: The system downloads the image and executes it. The image contains a small iPXE script, which makes iPXE retrieve its actual -configuration from a web service running (usually) on ``boot00.psi.ch``. +configuration from a web service running (usually) on ``sysdb.psi.ch``. Specifically, it queries the URL -``https://boot00.psi.ch/ipxe/v1/config?mac=``, where ```` is the MAC +``https://sysdb.psi.ch/ipxe/v1/config?mac=``, where ```` is the MAC address of the interface used by iPXE. -The web service on ``boot00.psi.ch`` will generate the iPXE configuration on the +The web service on ``sysdb.psi.ch`` will generate the iPXE configuration on the fly, depending on whether the system is supposed to be reinstalled and if so, which distribution it is supposed to use. The menu offers other options as well, e.g. an interactive iPXE shell and a memory test. @@ -68,4 +68,4 @@ roughly like the following:: :dhcp_succeeded - chain http://boot00.psi.ch/ipxe/v1/config?mac=${netX/mac} + chain http://sysdb.psi.ch/ipxe/v1/config?mac=${netX/mac} diff --git a/admin-guide/deployment/kickstart.rst b/admin-guide/deployment/kickstart.rst index a4673469..2526a20c 100644 --- a/admin-guide/deployment/kickstart.rst +++ b/admin-guide/deployment/kickstart.rst @@ -3,7 +3,7 @@ Kickstart The auto-generated Grub or iPXE configuration will, when installing RHEL, instruct the RHEL installer to download the Kickstart configuration from -``https://boot00.psi.ch/kickstart/v1/config?fqdn=&instrepo=``, +``https://sysdb.psi.ch/kickstart/v1/config?fqdn=&instrepo=``, where ```` is the FQDN of the host to be installed, and ```` is the installation repository to be used. diff --git a/admin-guide/mgmt-tools/bob.rst b/admin-guide/mgmt-tools/bob.rst index 58493ac0..89e7f656 100644 --- a/admin-guide/mgmt-tools/bob.rst +++ b/admin-guide/mgmt-tools/bob.rst @@ -5,59 +5,7 @@ bob - a CLI sysdb client retrieving information about environments, nodes, MAC addresses and attributes. It can authenticate using Kerberos credentials or username/password pairs. - -Installation and setup ----------------------- - -For production use of bob there will be RPMs for bob and its dependencies. - -For the moment the steps are the following:: - - yum -y install python-setuptools python-requests-kerberos - git clone git@git.psi.ch:linux-infra/admin-tools.git - cd admin-tools - python setup.py install - - -Configuration -------------- - -``bob`` takes a number of arguments, one of which is the base URL of the sysdb -instance. It can be passed via ``--url BASEURL`` on the command-line, or using -the environment variable ``PSI_BOB_URL``. Therefore it is useful include the -following in your ``.bash_profile``:: - - PSI_BOB_URL=https://boot00.psi.ch/ - -Development ------------ -For development, the easiest way to use bob is the following:: - - pip install --user click - git clone XXX - cd YYY - pip install --user --editable . - -This will place a script ``bob`` in ``~/.local/bin``, which will reference the -code in the working directory of the repository clone. This way, local changes -in the source are effective immediately. It might be a good idea to use -``virtualenv`` to avoid installing dependencies system-wide. - -In addition to the ``PSI_BOB_URL`` it can be handy to also have the following -in your ``.bash_profile``:: - - alias bobtest='PSI_BOB_URL=https://boot00-test.psi.ch/ bob' - alias bobdev='PSI_BOB_URL=http://localhost:5000/ bob' - -During development sysdb usually listens on ``localhost:5000`` and doesn't use -SSL, explaining the second alias. It also doesn't support authentication, -instead expecting to find the username in the ``REMOTE_USER`` header. The latter -can be set using the ``--fake-user`` option in bob or using the ``-H`` option in -cURL:: - - bobdev --fake-user kaminski_k node set-attr foo.psi.ch ipxe_installer=rhel72server - - curl -X PUT -H REMOTE_USER:talamo_i http://localhost:5000/sysdb/v1/ +For more information about bob refer to https://git.psi.ch/linux-infra/bob Attributes ---------- diff --git a/admin-guide/mgmt-tools/sysdb.rst b/admin-guide/mgmt-tools/sysdb.rst index 24992d6b..cbc94e05 100644 --- a/admin-guide/mgmt-tools/sysdb.rst +++ b/admin-guide/mgmt-tools/sysdb.rst @@ -15,6 +15,5 @@ including #. arbitrary key-value pairs which can be used to store additional information like the Puppet role of a system. -sysdb runs on the boot server, ie. ``boot00.psi.ch`` (production) and -``boot00-test.psi.ch`` (testing). +sysdb runs on the server, ie. ``sysdb.psi.ch`` (production). diff --git a/admin-guide/software.rst b/admin-guide/software.rst index de977f3d..7e15c1b6 100644 --- a/admin-guide/software.rst +++ b/admin-guide/software.rst @@ -23,9 +23,7 @@ One thing to keep in mind is that several groups at PSI use the Satellite server directly, so not all PSI hosts known to the Satellite belong to the central Linux environment. In particular the network team has a number of systems there. -Normally only certain infrastructure systems are registered with the Satellite, -at this point the Puppet server (``puppet01.psi.ch``), the repository server -(``repo00.psi.ch``), and the boot server (``boot00.psi.ch``). +Normally only certain infrastructure systems are registered with the Satellite. About once a year we report the total number of systems to ETHZ, so they can track subscription usage. When we started using the ETHZ subscriptions in 2016 diff --git a/admin-guide/troubleshooting/deployment.rst b/admin-guide/troubleshooting/deployment.rst index 29fca094..db09ff43 100644 --- a/admin-guide/troubleshooting/deployment.rst +++ b/admin-guide/troubleshooting/deployment.rst @@ -51,8 +51,8 @@ Kickstart Typical problems during the Kickstart phase: -1. The Kickstart file cannot be retrieved from the boot server - ``boot00.psi.ch``. Typically caused by incorrect sysdb entries or firewalls. +1. The Kickstart file cannot be retrieved from the sysdb server + ``sysdb.psi.ch``. Typically caused by incorrect sysdb entries or firewalls. 2. Partitioning fails. This can happen because a) No disk is recognized, or the wrong disk is used diff --git a/infrastructure-guide/infrastructure_systems.md b/infrastructure-guide/infrastructure_systems.md index e611cb0f..3dc7f332 100644 --- a/infrastructure-guide/infrastructure_systems.md +++ b/infrastructure-guide/infrastructure_systems.md @@ -3,9 +3,9 @@ List of systems and their primary role: __Core Infrastructure:__ -* [pxeserv01](pxeserv01) - boot.psi.ch - 129.129.190.59 - TFTP server for PXE booting +* [boot.psi.ch](pxeserv01) - TFTP server for PXE booting -* [boot00](boot00) - sysdb.psi.ch - 129.129.160.210 - Runs sysdb, providing the dynamic iPXE, Grub and kickstart files +* [sysdb.psi.ch](sysdb_server) - Runs sysdb, providing the dynamic iPXE, Grub and kickstart files * [puppet01](puppet01) - puppet.psi.ch - 129.129.160.118 - Runs the puppet server for the RHEL7 infra @@ -40,7 +40,7 @@ __Enduser Systems__ ## Metrics -* [Overview Infrastructure](https://metrics.psi.ch/d/1SL13Nxmz/gfa-linux-tabular?orgId=1&from=now-6h&to=now&refresh=30s&var-env=telegraf_pli&var-host=boot00.psi.ch&var-host=influx00.psi.ch&var-host=lxweb00.psi.ch&var-host=metrics00.psi.ch&var-host=puppet01.psi.ch&var-host=pxeserv01.psi.ch&var-host=repo00.psi.ch&var-host=reposync.psi.ch) +* [Overview Infrastructure](https://metrics.psi.ch/d/1SL13Nxmz/gfa-linux-tabular?orgId=1&from=now-6h&to=now&refresh=30s&var-env=telegraf_pli&var-host=influx00.psi.ch&var-host=lxweb00.psi.ch&var-host=metrics00.psi.ch&var-host=puppet01.psi.ch&var-host=repo00.psi.ch&var-host=reposync.psi.ch) # Procedures diff --git a/infrastructure-guide/newbob.md b/infrastructure-guide/newbob.md index 27b7a09e..92676656 100644 --- a/infrastructure-guide/newbob.md +++ b/infrastructure-guide/newbob.md @@ -1,7 +1,7 @@ # How to grand a person access to bob/sysdb -bob is making http calls to the sysdb app. Authorization (https://git.psi.ch/linux-infra/sysdb#authentication-and-authorization) is done via krb5 tokens. Operations outside of environments (creating/changing the owner of/deleting environments) needs to be done by a sysdb admin, ie someone who is a member of the group sysdb-admins. Group membership of the authenticated users is evaluated on the OS level on boot00. So group memberships can be set both locally or in the AD. This makes it a bit confusing, but both are used. -The sysdb-admins specifically is a local group, see boot00:/etc/group +bob is making http calls to the sysdb app. Authorization (https://git.psi.ch/linux-infra/sysdb#authentication-and-authorization) is done via krb5 tokens. Operations outside of environments (creating/changing the owner of/deleting environments) needs to be done by a sysdb admin, ie someone who is a member of the group sysdb-admins. Group membership of the authenticated users is evaluated on the OS level on sysdb.psi.ch. So group memberships can be set both locally or in the AD. This makes it a bit confusing, but both are used. +The sysdb-admins specifically is a local group, see /etc/group For the envs (bob env list), only adding and listing are implemented in bob, any other operation, like deletion or modification can only be performed in the sysdb sqlite database itself. diff --git a/infrastructure-guide/newver.md b/infrastructure-guide/newver.md index 83f45531..79e6152a 100644 --- a/infrastructure-guide/newver.md +++ b/infrastructure-guide/newver.md @@ -16,12 +16,12 @@ https://git.psi.ch/linux-infra/sysdb/blob/prod/sysdb/ipxe_template.py https://git.psi.ch/linux-infra/sysdb/blob/prod/sysdb/grub_template.py -Once the change is committed, the changes have to be pulled on boot00: +Once the change is committed, the changes have to be pulled on sysdb.psi.ch: ``` -[root@boot00 ~]# cd /var/www/sysdb/app/ -[root@boot00 app]# git pull -[root@boot00 app]# systemctl restart httpd +[root@sysdb ~]# cd /var/www/sysdb/app/ +[root@sysdb ~]# git pull +[root@sysdb ~]# systemctl restart httpd ``` The changes only come live after a restart of the httpd. \ No newline at end of file diff --git a/infrastructure-guide/overview_linux.drawio.svg b/infrastructure-guide/overview_linux.drawio.svg index acb8d0a8..d4e7cc9a 100644 --- a/infrastructure-guide/overview_linux.drawio.svg +++ b/infrastructure-guide/overview_linux.drawio.svg @@ -1,4 +1,4 @@ - + @@ -28,13 +28,13 @@
- cron / rsync (every minute) + timer / pull (every 30s)
- cron / rsy... + timer / pu... @@ -50,7 +50,7 @@
@@ -219,15 +219,17 @@
- boot00.psi.ch -
+ sysdb.psi.ch
+ + lx-sysdb-01.psi.ch +
- boot00.psi.ch... + sysdb.psi.ch... @@ -237,19 +239,19 @@
-
- +
+ Services: -
- +
+ sysdb
-
+
- serves grub config @@ -308,15 +310,15 @@
- + repo sync -
- +
+ sources defined in -
- +
+ /opt/pli/etc/yum.conf
@@ -394,12 +396,12 @@
- pxeserv01.psi.ch + boot.psi.ch
- 129.129.190.59 + lx-boot-01.psi.ch
@@ -407,7 +409,7 @@
- pxeserv01.psi.ch... + boot.psi.ch... @@ -434,27 +436,21 @@ -
+
-

+

- /afs/psi.ch/service/linux/tftpboot/ + https://git.psi.ch/linux-infra/network-boot

-

- - /afs/psi.ch/service/linux/tftpboot - - /pxelinux.cfg -

- /afs/psi.c... + https://git.psi.ch/linux-infra/network-boot @@ -465,7 +461,7 @@
-

+

/afs/psi.ch/service/linux/ @@ -564,12 +560,12 @@

-
+
repo00.psi.ch
-
+
129.129.160.212 @@ -770,10 +766,10 @@
-

+

Local storage for data

-

+

/var/lib/influxdb

@@ -792,40 +788,40 @@
-

+

/dist "/afs/psi.ch/project/linux/www/dist"

-

+

/kickstart "/afs/psi.ch/project/linux/www/kickstart"

-

+

/mirror "/afs/psi.ch/project/linux/www/mirror"

-

+

/pxe "/afs/psi.ch/service/linux/tftpboot"

-

+

/ext/cpt "/afs/psi.ch/project/cpt/repo/"

-

+

/ext/gfa "/afs/psi.ch/project/gfa-controls-sw-repo"

-

+

/ext/gpfs "/afs/psi.ch/software/linux/dist/scientificlinux/7x/x86_64/GPFS"

-

+

/ext/hpc-extra "/afs/psi.ch/software/linux/dist/scientificlinux/7x/x86_64/hpc-extra"

-

+

/ext/lmu "/afs/psi.ch/project/lmu/lmu_rpm/"

-

+

/ext/ofed "/afs/psi.ch/software/linux/dist/scientificlinux/7x/x86_64/OFED"

-

+

/ext/slurm "/afs/psi.ch/software/linux/dist/scientificlinux/7x/x86_64/slurm"

-

+

/ext/tier3 "/afs/psi.ch/software/linux/dist/scientific/6/tier3"

@@ -888,26 +884,6 @@ - - - - -
-
-
- boot00-test.psi.ch -
-
-
-
- - - boot00-test.psi.ch - - - - - @@ -1014,7 +990,7 @@ - Viewer does not support full SVG 1.1 + Text is not SVG - cannot display diff --git a/infrastructure-guide/boot00.md b/infrastructure-guide/sysdb_server.md similarity index 100% rename from infrastructure-guide/boot00.md rename to infrastructure-guide/sysdb_server.md