32 Commits

Author SHA1 Message Date
Ralph Lange
e10326b129 Update top README (reference module list, version number)
Some checks failed
ci-scripts build/test / clang / debug / ubuntu-16.04 (push) Has been cancelled
ci-scripts build/test / clang / debug / ubuntu-18.04 (push) Has been cancelled
ci-scripts build/test / clang / debug / ubuntu-20.04 (push) Has been cancelled
ci-scripts build/test / clang / default / ubuntu-16.04 (push) Has been cancelled
ci-scripts build/test / clang / default / ubuntu-18.04 (push) Has been cancelled
ci-scripts build/test / clang / default / ubuntu-20.04 (push) Has been cancelled
ci-scripts build/test / clang / static / ubuntu-16.04 (push) Has been cancelled
ci-scripts build/test / clang / static / ubuntu-18.04 (push) Has been cancelled
ci-scripts build/test / clang / static / ubuntu-20.04 (push) Has been cancelled
ci-scripts build/test / clang / static-debug / ubuntu-18.04 (push) Has been cancelled
ci-scripts build/test / clang / static-debug / ubuntu-20.04 (push) Has been cancelled
ci-scripts build/test / Unit tests on macos-10.15 (push) Has been cancelled
ci-scripts build/test / Unit tests on ubuntu-16.04 (push) Has been cancelled
ci-scripts build/test / Unit tests on ubuntu-18.04 (push) Has been cancelled
ci-scripts build/test / Unit tests on ubuntu-20.04 (push) Has been cancelled
ci-scripts build/test / Unit tests on windows-2016 (push) Has been cancelled
ci-scripts build/test / Unit tests on windows-2019 (push) Has been cancelled
ci-scripts build/test / clang / static-debug / ubuntu-16.04 (push) Has been cancelled
ci-scripts build/test / gcc / default / ubuntu-18.04 (push) Has been cancelled
ci-scripts build/test / gcc / default / ubuntu-20.04 (push) Has been cancelled
ci-scripts build/test / gcc / static / ubuntu-16.04 (push) Has been cancelled
ci-scripts build/test / gcc / static / ubuntu-18.04 (push) Has been cancelled
ci-scripts build/test / gcc / static / ubuntu-20.04 (push) Has been cancelled
ci-scripts build/test / gcc / static-debug / ubuntu-16.04 (push) Has been cancelled
ci-scripts build/test / gcc / static-debug / ubuntu-18.04 (push) Has been cancelled
ci-scripts build/test / gcc / static-debug / ubuntu-20.04 (push) Has been cancelled
ci-scripts build/test / gcc / debug / ubuntu-16.04 (push) Has been cancelled
ci-scripts build/test / gcc / debug / ubuntu-18.04 (push) Has been cancelled
ci-scripts build/test / gcc / debug / ubuntu-20.04 (push) Has been cancelled
ci-scripts build/test / gcc / default / ubuntu-16.04 (push) Has been cancelled
ci-scripts build/test / gcc / default / windows-2016 (push) Has been cancelled
ci-scripts build/test / gcc / default / windows-2019 (push) Has been cancelled
ci-scripts build/test / clang / debug / macos-10.15 (push) Has been cancelled
ci-scripts build/test / clang / default / macos-10.15 (push) Has been cancelled
ci-scripts build/test / gcc / debug / windows-2016 (push) Has been cancelled
ci-scripts build/test / gcc / debug / windows-2019 (push) Has been cancelled
ci-scripts build/test / gcc / static / windows-2016 (push) Has been cancelled
ci-scripts build/test / gcc / static / windows-2019 (push) Has been cancelled
ci-scripts build/test / gcc / static-debug / windows-2016 (push) Has been cancelled
ci-scripts build/test / gcc / static-debug / windows-2019 (push) Has been cancelled
ci-scripts build/test / vs2017 / debug / windows-2016 (push) Has been cancelled
ci-scripts build/test / vs2017 / default / windows-2016 (push) Has been cancelled
ci-scripts build/test / vs2017 / static / windows-2016 (push) Has been cancelled
ci-scripts build/test / vs2017 / static-debug / windows-2016 (push) Has been cancelled
ci-scripts build/test / vs2019 / debug / windows-2019 (push) Has been cancelled
ci-scripts build/test / vs2019 / default / windows-2019 (push) Has been cancelled
ci-scripts build/test / vs2019 / static / windows-2019 (push) Has been cancelled
ci-scripts build/test / vs2019 / static-debug / windows-2019 (push) Has been cancelled
ci-scripts build/test / RTEMS4.10 / debug / ubuntu-18.04 (push) Has been cancelled
ci-scripts build/test / RTEMS4.9 / debug / ubuntu-18.04 (push) Has been cancelled
ci-scripts build/test / RTEMS4.10 / default / ubuntu-18.04 (push) Has been cancelled
ci-scripts build/test / RTEMS4.9 / default / ubuntu-18.04 (push) Has been cancelled
ci-scripts build/test / RTEMS4.10 / static / ubuntu-18.04 (push) Has been cancelled
ci-scripts build/test / RTEMS4.9 / static / ubuntu-18.04 (push) Has been cancelled
ci-scripts build/test / RTEMS4.10 / static-debug / ubuntu-18.04 (push) Has been cancelled
ci-scripts build/test / RTEMS4.9 / static-debug / ubuntu-18.04 (push) Has been cancelled
ci-scripts build/test / WINE32 / debug / ubuntu-18.04 (push) Has been cancelled
ci-scripts build/test / WINE64 / debug / ubuntu-18.04 (push) Has been cancelled
ci-scripts build/test / WINE32 / default / ubuntu-18.04 (push) Has been cancelled
ci-scripts build/test / WINE64 / default / ubuntu-18.04 (push) Has been cancelled
ci-scripts build/test / WINE32 / static / ubuntu-18.04 (push) Has been cancelled
ci-scripts build/test / WINE64 / static / ubuntu-18.04 (push) Has been cancelled
ci-scripts build/test / WINE32 / static-debug / ubuntu-18.04 (push) Has been cancelled
ci-scripts build/test / WINE64 / static-debug / ubuntu-18.04 (push) Has been cancelled
2020-06-29 17:32:36 +02:00
Ralph Lange
bd7612dcab Add install of 'p7zip-full' to Travis configuration and docs 2020-06-29 16:51:36 +02:00
Ralph Lange
1b066feacc Special treatment for .patch and .zip/.7z HOOK files
(closes #44)
2020-06-29 15:43:11 +02:00
Ralph Lange
ee362d84aa gh-actions: set 'on:' filter to build everything
- push and pull_request notifications for all branches
2020-06-29 12:13:28 +02:00
Ralph Lange
9942f81988 Fix link in top README 2020-06-29 12:02:26 +02:00
Ralph Lange
ce99ab4e52 Merge pull request #52 from ralphlange/devel/gh-actions
Add support for GitHub Actions
2020-06-29 10:45:23 +02:00
Ralph Lange
08d2f23295 Update README files; add gh-action example files 2020-06-29 10:44:15 +02:00
Ralph Lange
a439d7aa64 cue: add fold control for gh-actions 2020-06-29 10:44:15 +02:00
Ralph Lange
6eefee8d18 gh-actions: add jobs for WINE cross builds 2020-06-29 10:44:15 +02:00
Ralph Lange
c7a134e1fb cue: use sudo when unpacking rtems cross compiler (WA)
- not needed on Travis (where target directory is under HOME)
2020-06-29 10:44:14 +02:00
Ralph Lange
ba5173a494 gh-actions: add jobs for RTEMS cross builds 2020-06-29 10:44:14 +02:00
Ralph Lange
b568ca982b cue: set PERL to Strawberry on gh-actions/windows 2020-06-29 10:44:14 +02:00
Ralph Lange
04f99fe6c4 gh-actions: add jobs for clang; failfast = false 2020-06-29 10:44:14 +02:00
Ralph Lange
0b54ff61c4 gh-actions: add configuration; split linux/macos/windows builds 2020-06-29 10:44:14 +02:00
Ralph Lange
074a7ab617 gh-actions: add windows-2016, ubuntu-20.04 jobs 2020-06-29 10:44:14 +02:00
Ralph Lange
5a0acc2228 cue: quiet down chocolatey installation 2020-06-29 10:44:14 +02:00
Ralph Lange
1c0f786eeb gh-actions: install re2c from BREW 2020-06-29 10:44:14 +02:00
Ralph Lange
4b76e0623d cue: install Homebrew packages from BREW on macOS 2020-06-29 10:44:14 +02:00
Ralph Lange
e8508cb658 gh-actions: add macOS jobs 2020-06-29 10:44:14 +02:00
Ralph Lange
388cc57533 gh-actions: improve naming 2020-06-29 10:44:14 +02:00
Ralph Lange
5beeada73f cue: improve logging in setup_for_build() 2020-06-29 10:44:14 +02:00
Ralph Lange
348a7dc851 cue-test: set platform/compiler correctly for windows tests 2020-06-29 10:44:14 +02:00
Ralph Lange
ab52808c63 cue-test: add 'findvs' action
walks the file system to find 'vcvarsall.bat'
2020-06-29 10:44:14 +02:00
Ralph Lange
3a99f3c9d6 cue: add vcvarsall.bat location for vs2019/Enterprise
as used by GitHub Actions windows-2019 runner
2020-06-29 10:44:14 +02:00
Ralph Lange
6c645363b3 gh-actions: add vs2019 build on windows 2020-06-29 10:44:13 +02:00
Ralph Lange
2dbd777852 gh-actions: improve workflow naming; choco install re2c 2020-06-29 10:44:13 +02:00
Ralph Lange
576b5a4caf cue: fix adding Strawberry Perl to PATH 2020-06-29 10:44:13 +02:00
Ralph Lange
031a756941 gh-actions: move environment setting to workflow level 2020-06-29 10:44:13 +02:00
Ralph Lange
f324cd3bdd Add proper detection for github-actions service 2020-06-29 10:44:13 +02:00
Ralph Lange
0ba02c5b03 gh-actions: initial build-and-test.yml
runs unit tests and a simple gcc build on Linux and Windows
2020-06-29 10:44:13 +02:00
Ralph Lange
b0c910c1cc appveyor: don't trigger build on changes in .github/workflow 2020-06-29 10:44:13 +02:00
Ralph Lange
59304f83c2 Use patch to add RTEMS-pc386-qemu target to older base 2020-06-29 10:44:13 +02:00
15 changed files with 682 additions and 91 deletions

View File

@@ -37,6 +37,7 @@ skip_commits:
- '**/*.html' - '**/*.html'
- '**/*.md' - '**/*.md'
- '.travis.yml' - '.travis.yml'
- '.github/workflows/*'
#---------------------------------# #---------------------------------#
# additional packages # # additional packages #

154
.github/workflows/build-and-test.yml vendored Normal file
View File

@@ -0,0 +1,154 @@
name: ci-scripts build/test
on: [push, pull_request]
env:
SETUP_PATH: .:.ci
SET: test01
VV: 1
BASE_RECURSIVE: NO
CMP: gcc
APT: re2c
CHOCO: re2c
BREW: re2c
jobs:
unit-test:
name: Unit tests on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-20.04, ubuntu-18.04, ubuntu-16.04, windows-2019, windows-2016, macos-10.15]
steps:
- uses: actions/checkout@v2
- name: Show initial environment
run: python cue-test.py env
- name: Run unit tests
run: python cue-test.py
build-linux:
name: ${{ matrix.cmp }} / ${{ matrix.configuration }} / ${{ matrix.os }}
runs-on: ${{ matrix.os }}
env:
CMP: ${{ matrix.cmp }}
BCFG: ${{ matrix.configuration }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-20.04, ubuntu-18.04, ubuntu-16.04]
cmp: [gcc, clang]
configuration: [default, static, debug, static-debug]
steps:
- uses: actions/checkout@v2
- name: Prepare and compile dependencies
run: python cue.py prepare
- name: Build main module (example app)
run: python cue.py build
- name: Run main module tests
run: python cue.py test
- name: Collect and show test results
run: python cue.py test-results
build-macos:
name: ${{ matrix.cmp }} / ${{ matrix.configuration }} / ${{ matrix.os }}
runs-on: ${{ matrix.os }}
env:
CMP: ${{ matrix.cmp }}
BCFG: ${{ matrix.configuration }}
strategy:
fail-fast: false
matrix:
os: [macos-10.15]
cmp: [clang]
configuration: [default, debug]
steps:
- uses: actions/checkout@v2
- name: Prepare and compile dependencies
run: python cue.py prepare
- name: Build main module (example app)
run: python cue.py build
- name: Run main module tests
run: python cue.py test
- name: Collect and show test results
run: python cue.py test-results
build-windows:
name: ${{ matrix.cmp }} / ${{ matrix.configuration }} / ${{ matrix.os }}
runs-on: ${{ matrix.os }}
env:
CMP: ${{ matrix.cmp }}
BCFG: ${{ matrix.configuration }}
strategy:
fail-fast: false
matrix:
os: [windows-2019, windows-2016]
cmp: [gcc, vs2019, vs2017]
configuration: [default, static, debug, static-debug]
exclude:
- os: windows-2019
cmp: vs2017
- os: windows-2016
cmp: vs2019
steps:
- uses: actions/checkout@v2
- name: Prepare and compile dependencies
run: python cue.py prepare
- name: Build main module (example app)
run: python cue.py build
- name: Run main module tests
run: python cue.py test
- name: Collect and show test results
run: python cue.py test-results
build-rtems:
name: RTEMS${{ matrix.rtems }} / ${{ matrix.configuration }} / ${{ matrix.os }}
runs-on: ${{ matrix.os }}
env:
CMP: ${{ matrix.cmp }}
BCFG: ${{ matrix.configuration }}
RTEMS: ${{ matrix.rtems }}
APT: re2c g++-mingw-w64-i686 g++-mingw-w64-x86-64 qemu-system-x86
strategy:
fail-fast: false
matrix:
os: [ubuntu-18.04]
cmp: [gcc]
configuration: [default, static, debug, static-debug]
rtems: ["4.9", "4.10"]
steps:
- uses: actions/checkout@v2
- name: Prepare and compile dependencies
run: python cue.py prepare
- name: Build main module (example app)
run: python cue.py build
- name: Run main module tests
run: python cue.py test
- name: Collect and show test results
run: python cue.py test-results
build-wine:
name: WINE${{ matrix.wine }} / ${{ matrix.configuration }} / ${{ matrix.os }}
runs-on: ${{ matrix.os }}
env:
CMP: ${{ matrix.cmp }}
BCFG: ${{ matrix.configuration }}
WINE: ${{ matrix.wine }}
APT: re2c g++-mingw-w64-i686 g++-mingw-w64-x86-64
strategy:
fail-fast: false
matrix:
os: [ubuntu-18.04]
cmp: [gcc]
configuration: [default, static, debug, static-debug]
wine: [32, 64]
steps:
- uses: actions/checkout@v2
- name: Prepare and compile dependencies
run: python cue.py prepare
- name: Build main module (example app)
run: python cue.py build
- name: Run main module tests
run: python cue.py test
- name: Collect and show test results
run: python cue.py test-results

View File

@@ -30,9 +30,11 @@ addons:
- g++-mingw-w64-i686 - g++-mingw-w64-i686
- g++-mingw-w64-x86-64 - g++-mingw-w64-x86-64
- qemu-system-x86 - qemu-system-x86
- p7zip-full
homebrew: homebrew:
packages: packages:
- re2c - re2c
- p7zip
update: true update: true
install: install:

View File

@@ -1,6 +1,7 @@
<a target="_blank" href="http://semver.org">![Version][badge.version]</a> <a target="_blank" href="http://semver.org">![Version][badge.version]</a>
<a target="_blank" href="https://travis-ci.org/epics-base/ci-scripts">![Travis status][badge.travis]</a> <a target="_blank" href="https://travis-ci.org/epics-base/ci-scripts">![Travis status][badge.travis]</a>
<a target="_blank" href="https://ci.appveyor.com/project/epics-base/ci-scripts">![AppVeyor status][badge.appveyor]</a> <a target="_blank" href="https://ci.appveyor.com/project/epics-base/ci-scripts">![AppVeyor status][badge.appveyor]</a>
<a target="_blank" href="https://github.com/epics-base/ci-scripts/actions">![GitHub Actions status][badge.gh-actions]</a>
# Continuous Integration for EPICS Modules # Continuous Integration for EPICS Modules
@@ -58,12 +59,12 @@ levels as the example files.
### [Travis-CI](https://travis-ci.org/) ### [Travis-CI](https://travis-ci.org/)
- Five parallel runners on Linux/Windows (one runner on MacOS) - Five parallel runners on Linux/Windows (one runner on MacOS)
- Use different compilers (gcc, clang) - Ubuntu 12/14/16/18, MacOS 10.13, Windows Server v1809
- Use different gcc versions - Compile natively on Linux (different versions of gcc, clang)
- Cross-compile for Windows 32bit and 64bit using MinGW and WINE
- Cross-compile for RTEMS 4.9 and 4.10 (Base >= 3.15)
- Compile natively on MacOS (clang) - Compile natively on MacOS (clang)
- Compile natively on Windows (gcc/MinGW, Visual Studio 2017) - Compile natively on Windows (gcc/MinGW, Visual Studio 2017)
- Cross-compile for Windows 32bit and 64bit using MinGW and WINE
- Cross-compile for RTEMS 4.9 and 4.10 (Base >= 3.15)
- Built dependencies are cached (for faster builds). - Built dependencies are cached (for faster builds).
See specific See specific
@@ -72,8 +73,8 @@ for more details.
### [AppVeyor](https://www.appveyor.com/) ### [AppVeyor](https://www.appveyor.com/)
- One parallel runner (all builds are sequential) - One parallel runner (all builds are sequential)
- Use different compilers (Visual Studio, gcc/MinGW) - Windows Server 2012/2016/2019
- Use different Visual Studio versions: \ - Compile using gcc/MinGW or different Visual Studio versions: \
2008, 2010, 2012, 2013, 2015, 2017, 2019 2008, 2010, 2012, 2013, 2015, 2017, 2019
- Compile for Windows 32bit and 64bit - Compile for Windows 32bit and 64bit
- No useful caching available. - No useful caching available.
@@ -82,15 +83,30 @@ See specific
**[ci-scripts on AppVeyor README](appveyor/README.md)** **[ci-scripts on AppVeyor README](appveyor/README.md)**
for more details. for more details.
### [GitHub Actions](https://github.com/)
- 20 parallel runners on Linux/Windows (5 runners on MacOS)
- Ubuntu 16/18/20, MacOS 10.15, Windows Server 2016/2019
- Compile natively on Linux (gcc, clang)
- Compile natively on MacOS (clang)
- Compile natively on Windows (gcc/MinGW, Visual Studio 2017 & 2019)
- Cross-compile for Windows 32bit and 64bit using MinGW and WINE
- Cross-compile for RTEMS 4.9 and 4.10 (Base >= 3.15)
- Caching not supported by ci-scripts yet.
See specific
**[ci-scripts on GitHub Actions README](github-actions/README.md)**
for more details.
## How to Use the CI-Scripts ## How to Use the CI-Scripts
1. Get an account on a supported CI service provider platform. 1. Get an account on a supported CI service provider platform
(e.g. [Travis-CI](https://travis-ci.org/), (e.g. [Travis-CI](https://travis-ci.org/),
[AppVeyor](https://www.appveyor.com/), ...) [AppVeyor](https://www.appveyor.com/), ...).
GitHub Actions does not require a separate account.
(More details in the specific README of the subdirectory.) (More details in the specific README of the subdirectory.)
2. In your Support Module, add this ci-scripts repository 2. In your module, add this ci-scripts repository
as a Git Submodule (name suggestion: `.ci`). as a Git Submodule (name suggestion: `.ci`).
```bash ```bash
git submodule add https://github.com/epics-base/ci-scripts .ci git submodule add https://github.com/epics-base/ci-scripts .ci
@@ -225,10 +241,11 @@ recursing into submodules. [default is including submodules: `YES`]
be always be extended by the release or branch name as `<name>-<version>`. be always be extended by the release or branch name as `<name>-<version>`.
[default is the slug in lower case: `foo`] [default is the slug in lower case: `foo`]
`FOO_HOOK=<script>` Set the name of a script that will be run after cloning `FOO_HOOK=<hook>` Set the name of a `.patch` file, a `.zip` or `.7z` archive
the module, before compiling it. Working directory when running the script or a script that will be applied (using `-p1`), extracted or run after cloning
is the root of the targeted module (e.g. `.../.cache/foo-1.2`). the module, before compiling it.
[default: no hooks are run] Working directory is the root of the targeted module,
e.g., `.../.cache/foo-1.2`). [default: no hook]
`FOO_VARNAME=<name>` Set the name that is used for the module when creating `FOO_VARNAME=<name>` Set the name that is used for the module when creating
the `RELEASE.local` files. [default is the slug in upper case: `FOO`] the `RELEASE.local` files. [default is the slug in upper case: `FOO`]
@@ -271,6 +288,8 @@ in the service specific subdirectories:
EPICS Modules: EPICS Modules:
[ASYN](https://github.com/epics-modules/asyn), [ASYN](https://github.com/epics-modules/asyn),
[autosave](https://github.com/epics-modules/autosave),
[busy](https://github.com/epics-modules/busy),
[devlib2](https://github.com/epics-modules/devlib2), [devlib2](https://github.com/epics-modules/devlib2),
[ecmc](https://github.com/epics-modules/ecmc), [ecmc](https://github.com/epics-modules/ecmc),
[gtest](https://github.com/epics-modules/gtest), [gtest](https://github.com/epics-modules/gtest),
@@ -279,10 +298,14 @@ EPICS Modules:
[MCoreUtils](https://github.com/epics-modules/MCoreUtils), [MCoreUtils](https://github.com/epics-modules/MCoreUtils),
[modbus](https://github.com/epics-modules/modbus), [modbus](https://github.com/epics-modules/modbus),
[motor](https://github.com/epics-modules/motor), [motor](https://github.com/epics-modules/motor),
[mrfioc2](https://github.com/epics-modules/mrfioc2),
[OPCUA](https://github.com/ralphlange/opcua), [OPCUA](https://github.com/ralphlange/opcua),
[PCAS](https://github.com/epics-modules/pcas), [PCAS](https://github.com/epics-modules/pcas),
[softGlueZync](https://github.com/epics-modules/softGlueZynq),
[sscan](https://github.com/epics-modules/sscan), [sscan](https://github.com/epics-modules/sscan),
[vac](https://github.com/epics-modules/vac) [std](https://github.com/epics-modules/std),
[vac](https://github.com/epics-modules/vac),
[xxx](https://github.com/epics-modules/xxx)
ESS: [EtherCAT MC Motor Driver][ref.ethercatmc] ESS: [EtherCAT MC Motor Driver][ref.ethercatmc]
@@ -342,16 +365,16 @@ This will make all builds (not just for your module) verbose.
Update the submodule in `.ci` first, then change your CI configuration Update the submodule in `.ci` first, then change your CI configuration
(if needed) and commit both to your module. E.g., to update your Travis (if needed) and commit both to your module. E.g., to update your Travis
setup to release 3.0.1 of ci-scripts: setup to release 3.1.0 of ci-scripts:
```bash ```bash
cd .ci cd .ci
git pull origin v3.0.1 git pull origin v3.1.0
cd - cd -
git add .ci git add .ci
# if needed: # if needed:
edit .travis.yml # and/or .appveyor.yml edit .travis.yml # and/or AppVeyor/GitHub Actions configuration
git add .travis.yml git add .travis.yml
git commit -m "Update ci-scripts submodule to v3.0.1" git commit -m "Update ci-scripts submodule to v3.1.0"
``` ```
Check the example configuration files inside ci-scripts (and their Check the example configuration files inside ci-scripts (and their
@@ -405,6 +428,7 @@ in file LICENSE that is included with this distribution.
[badge.version]: https://badge.fury.io/gh/epics-base%2Fci-scripts.svg [badge.version]: https://badge.fury.io/gh/epics-base%2Fci-scripts.svg
[badge.travis]: https://travis-ci.org/epics-base/ci-scripts.svg?branch=master [badge.travis]: https://travis-ci.org/epics-base/ci-scripts.svg?branch=master
[badge.appveyor]: https://ci.appveyor.com/api/projects/status/8b578alg974axvux?svg=true [badge.appveyor]: https://ci.appveyor.com/api/projects/status/8b578alg974axvux?svg=true
[badge.gh-actions]: https://github.com/epics-base/ci-scripts/workflows/ci-scripts%20build/test/badge.svg
[reddit.bash]: https://www.reddit.com/r/bash/comments/393oqv/why_is_the_version_of_bash_included_in_os_x_so_old/ [reddit.bash]: https://www.reddit.com/r/bash/comments/393oqv/why_is_the_version_of_bash_included_in_os_x_so_old/

View File

@@ -3,8 +3,8 @@
## Features ## Features
- One parallel runner (all builds are sequential) - One parallel runner (all builds are sequential)
- Use different compilers (Visual Studio, gcc/MinGW) - Windows Server 2012/2016/2019
- Use different Visual Studio versions: \ - Compile using gcc/MinGW or different Visual Studio versions: \
2008, 2010, 2012, 2013, 2015, 2017, 2019 2008, 2010, 2012, 2013, 2015, 2017, 2019
- Compile for Windows 32bit and 64bit - Compile for Windows 32bit and 64bit
- No useful caching available. - No useful caching available.

View File

@@ -31,6 +31,13 @@ if 'APPVEYOR' in os.environ:
elif re.match(r'^macOS', os.environ['APPVEYOR_BUILD_WORKER_IMAGE']): elif re.match(r'^macOS', os.environ['APPVEYOR_BUILD_WORKER_IMAGE']):
ci_os = 'osx' ci_os = 'osx'
if 'GITHUB_ACTIONS' in os.environ:
ci_service = 'github-actions'
if os.environ['RUNNER_OS'] == 'macOS':
ci_os = 'osx'
else:
ci_os = os.environ['RUNNER_OS'].lower()
def find_in_file(regex, filename): def find_in_file(regex, filename):
file = open(filename, "r") file = open(filename, "r")
@@ -324,8 +331,14 @@ class TestDefaultModuleURLs(unittest.TestCase):
@unittest.skipIf(ci_os != 'windows', 'VCVars test only applies to windows') @unittest.skipIf(ci_os != 'windows', 'VCVars test only applies to windows')
class TestVCVars(unittest.TestCase): class TestVCVars(unittest.TestCase):
def test_vcvars(self): def test_vcvars(self):
if ci_service == 'appveyor': if ci_service == 'travis':
os.environ['TRAVIS_COMPILER'] = 'vs2017'
else:
os.environ['CONFIGURATION'] = 'default' os.environ['CONFIGURATION'] = 'default'
if ci_service == 'github-actions' and os.environ['IMAGEOS'] == 'win16':
os.environ['CMP'] = 'vs2017'
else:
os.environ['CMP'] = 'vs2019'
cue.detect_context() cue.detect_context()
cue.with_vcvars('env') cue.with_vcvars('env')
@@ -676,7 +689,7 @@ class TestSetupForBuild(unittest.TestCase):
args = Namespace(paths=[]) args = Namespace(paths=[])
cue.building_base = True cue.building_base = True
if ci_os == 'windows': if ci_os == 'windows':
sp.check_call(['choco', 'install', 'make']) sp.check_call(['choco', 'install', 'make', 'strawberryperl', '-ry'])
def setUp(self): def setUp(self):
if ci_service == 'appveyor': if ci_service == 'appveyor':
@@ -726,10 +739,10 @@ class TestSetupForBuild(unittest.TestCase):
@unittest.skipIf(ci_os != 'windows', 'HostArchPlatform test only applies to windows') @unittest.skipIf(ci_os != 'windows', 'HostArchPlatform test only applies to windows')
def test_HostArchPlatform(self): def test_HostArchPlatform(self):
if ci_service == 'travis': if ci_service == 'appveyor':
platforms = ['x64']
else:
platforms = ['x86', 'x64'] platforms = ['x86', 'x64']
else:
platforms = ['x64']
for platform in platforms: for platform in platforms:
for cc in ['vs2019', 'gcc']: for cc in ['vs2019', 'gcc']:
cue.ci['platform'] = platform cue.ci['platform'] = platform
@@ -750,18 +763,17 @@ class TestSetupForBuild(unittest.TestCase):
self.assertTrue(re.search('-mingw$', os.environ['EPICS_HOST_ARCH']), self.assertTrue(re.search('-mingw$', os.environ['EPICS_HOST_ARCH']),
'EPICS_HOST_ARCH (found {0}) is not -mingw for {1} / {2}' 'EPICS_HOST_ARCH (found {0}) is not -mingw for {1} / {2}'
.format(os.environ['EPICS_HOST_ARCH'], cc, platform)) .format(os.environ['EPICS_HOST_ARCH'], cc, platform))
pattern = {'x86': 'mingw32', 'x64': 'mingw64'} if ci_service == 'appveyor':
self.assertTrue(re.search(pattern[platform], os.environ['PATH']), pattern = {'x86': 'mingw32', 'x64': 'mingw64'}
'Binary location for {0} not in PATH (found {1})' self.assertTrue(re.search(pattern[platform], os.environ['PATH']),
.format(pattern[platform], os.environ['PATH'])) 'Binary location for {0} not in PATH (found PATH = {1})'
.format(pattern[platform], os.environ['PATH']))
@unittest.skipIf(ci_os != 'windows', 'Strawberry perl test only applies to windows') @unittest.skipIf(ci_os != 'windows', 'Strawberry perl test only applies to windows')
def test_StrawberryInPathVS2019(self): def test_StrawberryInPath(self):
if 'APPVEYOR' in os.environ:
os.environ['CMP'] = 'vs2019'
cue.setup_for_build(self.args) cue.setup_for_build(self.args)
self.assertTrue(re.search('strawberry', os.environ['PATH'], flags=re.IGNORECASE), self.assertTrue(re.search('strawberry', os.environ['PATH'], flags=re.IGNORECASE),
'Strawberry Perl installed but location not in PATH (found {0})' 'Strawberry Perl location not in PATH (found PATH = {0})'
.format(os.environ['PATH'])) .format(os.environ['PATH']))
def setBase314(self, yesno): def setBase314(self, yesno):
@@ -817,6 +829,44 @@ class TestSetupForBuild(unittest.TestCase):
'Extra make arg [{0}] not set (expected "bla {0}", found "{1}")' 'Extra make arg [{0}] not set (expected "bla {0}", found "{1}")'
.format(ind, cue.extra_makeargs[ind])) .format(ind, cue.extra_makeargs[ind]))
class TestHooks(unittest.TestCase):
location = os.path.join(cue.cachedir, 'hook_test')
bla_file = os.path.join(location, 'bla.txt')
new_file = os.path.join(location, 'dd', 'new.txt')
def setUp(self):
if os.path.exists(self.location):
shutil.rmtree(self.location, onerror=cue.remove_readonly)
try:
os.makedirs(self.location)
except:
pass
with open(self.bla_file, 'w') as f:
f.write('''LINE1=YES
LINE2=NO''')
def test_patchfile(self):
hook = os.path.join(builddir, 'test.patch')
cue.apply_patch(hook, cwd=self.location)
line1_yes = False
with open(self.bla_file) as f:
if 'LINE1=YES' in f.read():
line1_yes = True
self.assertFalse(line1_yes, "Patch didn't change line in test file 'bla.txt'")
self.assertTrue(os.path.exists(self.new_file), "patch didn't add new file")
def test_archiveZip(self):
hook = os.path.join(builddir, 'test.zip')
cue.extract_archive(hook, cwd=self.location)
self.assertTrue(os.path.exists(self.new_file), "archive extract didn't add new file")
def test_archive7z(self):
hook = os.path.join(builddir, 'test.7z')
cue.extract_archive(hook, cwd=self.location)
self.assertTrue(os.path.exists(self.new_file), "archive extract didn't add new file")
if __name__ == "__main__": if __name__ == "__main__":
if 'VV' in os.environ and os.environ['VV'] == '1': if 'VV' in os.environ and os.environ['VV'] == '1':
logging.basicConfig(level=logging.DEBUG) logging.basicConfig(level=logging.DEBUG)
@@ -827,5 +877,14 @@ if __name__ == "__main__":
if sys.argv[1:] == ['env']: if sys.argv[1:] == ['env']:
# testing with_vcvars # testing with_vcvars
[print(K, '=', V) for K, V in os.environ.items()] [print(K, '=', V) for K, V in os.environ.items()]
elif ci_os == 'windows' and sys.argv[1:] == ['findvs']:
from fnmatch import fnmatch
print('Available Visual Studio versions')
for base in (r'C:\Program Files (x86)', r'C:\Program Files'):
for root, dirs, files in os.walk(base):
for fname in files:
if fnmatch(fname, 'vcvarsall.bat'):
print('Found', os.path.join(root, fname))
sys.stdout.flush()
else: else:
unittest.main() unittest.main()

188
cue.py
View File

@@ -1,5 +1,5 @@
#!/usr/bin/env python #!/usr/bin/env python
"""CI build script for Linux/MacOS/Windows on Travis/AppVeyor """CI build script for Linux/MacOS/Windows on Travis/AppVeyor/GitHub-Actions
""" """
from __future__ import print_function from __future__ import print_function
@@ -22,11 +22,10 @@ def detect_context():
ci['os'] = os.environ['TRAVIS_OS_NAME'] ci['os'] = os.environ['TRAVIS_OS_NAME']
ci['platform'] = 'x64' ci['platform'] = 'x64'
ci['compiler'] = os.environ['TRAVIS_COMPILER'] ci['compiler'] = os.environ['TRAVIS_COMPILER']
if ci['os'] == 'windows': ci['choco'] += ['strawberryperl']
ci['choco'] += ['strawberryperl'] if re.match(r'^vs', ci['compiler']):
if re.match(r'^vs', ci['compiler']): # Only Visual Studio 2017 available
# Only Visual Studio 2017 available ci['compiler'] = 'vs2017'
ci['compiler'] = 'vs2017'
if 'BCFG' in os.environ: if 'BCFG' in os.environ:
buildconfig = os.environ['BCFG'].lower() buildconfig = os.environ['BCFG'].lower()
@@ -43,6 +42,24 @@ def detect_context():
ci['compiler'] = os.environ['CMP'] ci['compiler'] = os.environ['CMP']
buildconfig = os.environ['CONFIGURATION'].lower() buildconfig = os.environ['CONFIGURATION'].lower()
if 'GITHUB_ACTIONS' in os.environ:
ci['service'] = 'github-actions'
if os.environ['RUNNER_OS'] == 'macOS':
ci['os'] = 'osx'
else:
ci['os'] = os.environ['RUNNER_OS'].lower()
ci['platform'] = 'x64'
if 'CMP' in os.environ:
ci['compiler'] = os.environ['CMP']
ci['choco'] += ['strawberryperl']
if 'BCFG' in os.environ:
buildconfig = os.environ['BCFG'].lower()
if re.search('static', buildconfig):
ci['static'] = True
if re.search('debug', buildconfig):
ci['debug'] = True
if 'STATIC' in os.environ: if 'STATIC' in os.environ:
print("{0}WARNING: Variable 'STATIC' not supported anymore; use 'BCFG' instead{1}" print("{0}WARNING: Variable 'STATIC' not supported anymore; use 'BCFG' instead{1}"
.format(ANSI_RED, ANSI_RESET)) .format(ANSI_RED, ANSI_RESET))
@@ -52,11 +69,6 @@ def detect_context():
.format(ANSI_RED, buildconfig, ANSI_RESET)) .format(ANSI_RED, buildconfig, ANSI_RESET))
sys.stdout.flush() sys.stdout.flush()
if re.search('static', buildconfig):
ci['static'] = True
if re.search('debug', buildconfig):
ci['debug'] = True
if ci['static']: if ci['static']:
ci['configuration'] = 'static' ci['configuration'] = 'static'
else: else:
@@ -74,6 +86,9 @@ def detect_context():
if 'APT' in os.environ: if 'APT' in os.environ:
ci['apt'].extend(os.environ['APT'].split()) ci['apt'].extend(os.environ['APT'].split())
if 'BREW' in os.environ:
ci['homebrew'].extend(os.environ['BREW'].split())
ci['test'] = True ci['test'] = True
if 'TEST' in os.environ and os.environ['TEST'].lower() == 'no': if 'TEST' in os.environ and os.environ['TEST'].lower() == 'no':
ci['test'] = False ci['test'] = False
@@ -106,6 +121,7 @@ is_make3 = False
has_test_results = False has_test_results = False
silent_dep_builds = True silent_dep_builds = True
do_recompile = False do_recompile = False
installed_7z = False
def clear_lists(): def clear_lists():
@@ -130,6 +146,7 @@ def clear_lists():
ci['scriptsdir'] = '' ci['scriptsdir'] = ''
ci['choco'] = ['make'] ci['choco'] = ['make']
ci['apt'] = [] ci['apt'] = []
ci['homebrew'] = []
clear_lists() clear_lists()
@@ -153,11 +170,16 @@ ANSI_CLEAR = "\033[0K"
# Travis log fold control # Travis log fold control
# from https://github.com/travis-ci/travis-rubies/blob/build/build.sh # from https://github.com/travis-ci/travis-rubies/blob/build/build.sh
# GitHub Actions fold control
# from https://github.com/actions/toolkit/blob/master/docs/commands.md#group-and-ungroup-log-lines
def fold_start(tag, title): def fold_start(tag, title):
if ci['service'] == 'travis': if ci['service'] == 'travis':
print('travis_fold:start:{0}{1}{2}{3}' print('travis_fold:start:{0}{1}{2}{3}'
.format(tag, ANSI_YELLOW, title, ANSI_RESET)) .format(tag, ANSI_YELLOW, title, ANSI_RESET))
elif ci['service'] == 'github-actions':
print('::group::{0}{1}{2}'
.format(ANSI_YELLOW, title, ANSI_RESET))
elif ci['service'] == 'appveyor': elif ci['service'] == 'appveyor':
print('{0}===== \\/ \\/ \\/ ===== START: {1} ====={2}' print('{0}===== \\/ \\/ \\/ ===== START: {1} ====={2}'
.format(ANSI_YELLOW, title, ANSI_RESET)) .format(ANSI_YELLOW, title, ANSI_RESET))
@@ -168,6 +190,9 @@ def fold_end(tag, title):
if ci['service'] == 'travis': if ci['service'] == 'travis':
print('\ntravis_fold:end:{0}\r' print('\ntravis_fold:end:{0}\r'
.format(tag), end='') .format(tag), end='')
elif ci['service'] == 'github-actions':
print('::endgroup::'
.format(ANSI_YELLOW, title, ANSI_RESET))
elif ci['service'] == 'appveyor': elif ci['service'] == 'appveyor':
print('{0}----- /\\ /\\ /\\ ----- END: {1} -----{2}' print('{0}----- /\\ /\\ /\\ ----- END: {1} -----{2}'
.format(ANSI_YELLOW, title, ANSI_RESET)) .format(ANSI_YELLOW, title, ANSI_RESET))
@@ -181,7 +206,7 @@ elif 'HOME' in os.environ:
homedir = os.getenv('HOME') homedir = os.getenv('HOME')
cachedir = os.path.join(homedir, '.cache') cachedir = os.path.join(homedir, '.cache')
toolsdir = os.path.join(homedir, '.tools') toolsdir = os.path.join(homedir, '.tools')
rtemsdir = os.path.join(homedir, '.rtems') rtemsdir = r'/home/travis/.rtems' # Preliminary, until the next generation of toolchain
if 'CACHEDIR' in os.environ: if 'CACHEDIR' in os.environ:
cachedir = os.environ['CACHEDIR'] cachedir = os.environ['CACHEDIR']
@@ -189,8 +214,10 @@ if 'CACHEDIR' in os.environ:
vcvars_table = { vcvars_table = {
# https://en.wikipedia.org/wiki/Microsoft_Visual_Studio#History # https://en.wikipedia.org/wiki/Microsoft_Visual_Studio#History
'vs2019': [r'C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat'], 'vs2019': [r'C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat',
r'C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Auxiliary\Build\vcvarsall.bat'],
'vs2017': [r'C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build\vcvarsall.bat', 'vs2017': [r'C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build\vcvarsall.bat',
r'C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\VC\Auxiliary\Build\vcvarsall.bat',
r'C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Auxiliary\Build\vcvarsall.bat'], r'C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Auxiliary\Build\vcvarsall.bat'],
'vs2015': [r'C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat'], 'vs2015': [r'C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat'],
'vs2013': [r'C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\vcvarsall.bat'], 'vs2013': [r'C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\vcvarsall.bat'],
@@ -377,6 +404,24 @@ def call_make(args=[], **kws):
sys.exit(exitcode) sys.exit(exitcode)
def apply_patch(file, **kws):
place = kws.get('cwd', os.getcwd())
print('Applying patch {0} in {1}'.format(file, place))
logger.debug("EXEC '%s' in %s", ' '.join(['patch', '-p1', '-i', file]), place)
sys.stdout.flush()
sp.check_call(['patch', '-p1', '-i', file], cwd=place)
logger.debug('EXEC DONE')
def extract_archive(file, **kws):
place = kws.get('cwd', os.getcwd())
print('Extracting archive {0} in {1}'.format(file, place))
logger.debug("EXEC '%s' in %s", ' '.join(['7z', 'x', '-aoa', '-bd', file]), place)
sys.stdout.flush()
sp.check_call(['7z', 'x', '-aoa', '-bd', file], cwd=place)
logger.debug('EXEC DONE')
def get_git_hash(place): def get_git_hash(place):
logger.debug("EXEC 'git log -n1 --pretty=format:%%H' in %s", place) logger.debug("EXEC 'git log -n1 --pretty=format:%%H' in %s", place)
sys.stdout.flush() sys.stdout.flush()
@@ -489,13 +534,19 @@ def add_dependency(dep):
with open(release, 'w') as fout: with open(release, 'w') as fout:
print('-include $(TOP)/../RELEASE.local', file=fout) print('-include $(TOP)/../RELEASE.local', file=fout)
# run hook if defined # Apply HOOK
if dep + '_HOOK' in setup: if dep + '_HOOK' in setup:
hook = os.path.join(place, setup[dep + '_HOOK']) hook = setup[dep + '_HOOK']
if os.path.exists(hook): hook_file = os.path.join(curdir, hook)
print('Running hook {0} in {1}'.format(setup[dep + '_HOOK'], place)) if os.path.exists(hook_file):
sys.stdout.flush() if re.match(r'.+\\.patch$', hook):
sp.check_call(hook, shell=True, cwd=place) apply_patch(hook_file, cwd=place)
elif re.match(r'.+\\.(zip|7z)$', hook):
extract_archive(hook_file, cwd=place)
else:
print('Running hook {0} in {1}'.format(hook, place))
sys.stdout.flush()
sp.check_call(hook_file, shell=True, cwd=place)
# write checked out commit hash to marker file # write checked out commit hash to marker file
head = get_git_hash(place) head = get_git_hash(place)
@@ -551,32 +602,34 @@ def setup_for_build(args):
global is_base314, has_test_results, is_make3 global is_base314, has_test_results, is_make3
dllpaths = [] dllpaths = []
logger.debug('Setting up the build environment')
if ci['os'] == 'windows': if ci['os'] == 'windows':
if ci['service'] == 'appveyor': if os.path.exists(r'C:\Strawberry\perl\bin'):
if ci['compiler'] == 'vs2019': # Put strawberry perl in front of the PATH (so that Git Perl is further behind)
# put strawberry perl in the PATH logger.debug('Adding Strawberry Perl in front of the PATH')
os.environ['PATH'] = os.pathsep.join([os.path.join(r'C:\Strawberry\perl\site\bin'), os.environ['PATH'] = os.pathsep.join([r'C:\Strawberry\c\bin',
os.path.join(r'C:\Strawberry\perl\bin'), r'C:\Strawberry\perl\site\bin',
os.environ['PATH']]) r'C:\Strawberry\perl\bin',
if ci['compiler'] == 'gcc':
if 'INCLUDE' not in os.environ:
os.environ['INCLUDE'] = ''
if ci['platform'] == 'x86':
os.environ['INCLUDE'] = os.pathsep.join(
[r'C:\mingw-w64\i686-6.3.0-posix-dwarf-rt_v5-rev1\mingw32\include',
os.environ['INCLUDE']])
os.environ['PATH'] = os.pathsep.join([r'C:\mingw-w64\i686-6.3.0-posix-dwarf-rt_v5-rev1\mingw32\bin',
os.environ['PATH']])
elif ci['platform'] == 'x64':
os.environ['INCLUDE'] = os.pathsep.join(
[r'C:\mingw-w64\x86_64-8.1.0-posix-seh-rt_v6-rev0\mingw64\include',
os.environ['INCLUDE']])
os.environ['PATH'] = os.pathsep.join([r'C:\mingw-w64\x86_64-8.1.0-posix-seh-rt_v6-rev0\mingw64\bin',
os.environ['PATH']])
if ci['service'] == 'travis':
os.environ['PATH'] = os.pathsep.join([r'C:\Strawberry\perl\site\bin', r'C:\Strawberry\perl\bin',
os.environ['PATH']]) os.environ['PATH']])
if ci['service'] == 'appveyor' and ci['compiler'] == 'gcc':
logger.debug('Adding AppVeyor MSYS2/MinGW installation to PATH and INCLUDE')
if 'INCLUDE' not in os.environ:
os.environ['INCLUDE'] = ''
if ci['platform'] == 'x86':
os.environ['INCLUDE'] = os.pathsep.join(
[r'C:\mingw-w64\i686-6.3.0-posix-dwarf-rt_v5-rev1\mingw32\include',
os.environ['INCLUDE']])
os.environ['PATH'] = os.pathsep.join([r'C:\mingw-w64\i686-6.3.0-posix-dwarf-rt_v5-rev1\mingw32\bin',
os.environ['PATH']])
elif ci['platform'] == 'x64':
os.environ['INCLUDE'] = os.pathsep.join(
[r'C:\mingw-w64\x86_64-8.1.0-posix-seh-rt_v6-rev0\mingw64\include',
os.environ['INCLUDE']])
os.environ['PATH'] = os.pathsep.join([r'C:\mingw-w64\x86_64-8.1.0-posix-seh-rt_v6-rev0\mingw64\bin',
os.environ['PATH']])
# Find BASE location # Find BASE location
if not building_base: if not building_base:
with open(os.path.join(cachedir, 'RELEASE.local'), 'r') as f: with open(os.path.join(cachedir, 'RELEASE.local'), 'r') as f:
@@ -588,6 +641,8 @@ def setup_for_build(args):
else: else:
places['EPICS_BASE'] = '.' places['EPICS_BASE'] = '.'
logger.debug('Using EPICS Base at %s', places['EPICS_BASE'])
detect_epics_host_arch() detect_epics_host_arch()
if ci['os'] == 'windows': if ci['os'] == 'windows':
@@ -611,6 +666,7 @@ def setup_for_build(args):
with open(cfg_base_version) as myfile: with open(cfg_base_version) as myfile:
if 'BASE_3_14=YES' in myfile.read(): if 'BASE_3_14=YES' in myfile.read():
is_base314 = True is_base314 = True
logger.debug('Check if EPICS Base is a 3.14 series: %s', is_base314)
if not is_base314: if not is_base314:
rules_build = os.path.join(places['EPICS_BASE'], 'configure', 'RULES_BUILD') rules_build = os.path.join(places['EPICS_BASE'], 'configure', 'RULES_BUILD')
@@ -623,6 +679,7 @@ def setup_for_build(args):
# Check make version # Check make version
if re.match(r'^GNU Make 3', sp.check_output(['make', '-v']).decode('ascii')): if re.match(r'^GNU Make 3', sp.check_output(['make', '-v']).decode('ascii')):
is_make3 = True is_make3 = True
logger.debug('Check if make is a 3.x series: %s', is_make3)
# apparently %CD% is handled automagically # apparently %CD% is handled automagically
os.environ['TOP'] = os.getcwd() os.environ['TOP'] = os.getcwd()
@@ -653,9 +710,9 @@ def fix_etc_hosts():
# 127.0.1.1 localhost localhost ip4-loopback # 127.0.1.1 localhost localhost ip4-loopback
# 127.0.0.1 localhost nettuno travis vagrant travis-job-.... # 127.0.0.1 localhost nettuno travis vagrant travis-job-....
logger.debug("EXEC sudo sed -ie '/^127\.0\.1\.1/ s|localhost\s*||g' /etc/hosts") logger.debug("EXEC sudo sed -ie '/^127\\.0\\.1\\.1/ s|localhost\\s*||g' /etc/hosts")
sys.stdout.flush() sys.stdout.flush()
exitcode = sp.call(['sudo', 'sed', '-ie', '/^127\.0\.1\.1/ s|localhost\s*||g', '/etc/hosts']) sp.call(['sudo', 'sed', '-ie', '/^127\\.0\\.1\\.1/ s|localhost\\s*||g', '/etc/hosts'])
logger.debug('EXEC DONE') logger.debug('EXEC DONE')
@@ -781,14 +838,17 @@ CROSS_COMPILER_TARGET_ARCHS += windows-x64-mingw''')
RTEMS_VERSION={0} RTEMS_VERSION={0}
RTEMS_BASE={1}'''.format(os.environ['RTEMS'], rtemsdir)) RTEMS_BASE={1}'''.format(os.environ['RTEMS'], rtemsdir))
# Base 3.15 doesn't have -qemu target architecture # Patch Base 3.15 that doesn't have -qemu target architecture
qemu_suffix = '' if not os.path.exists(os.path.join(places['EPICS_BASE'], 'configure', 'os',
if os.path.exists(os.path.join(places['EPICS_BASE'], 'configure', 'os',
'CONFIG.Common.RTEMS-pc386-qemu')): 'CONFIG.Common.RTEMS-pc386-qemu')):
qemu_suffix = '-qemu' print('Adding RTEMS-pc386-qemu target to Base in {0}'.format(places['EPICS_BASE']))
sys.stdout.flush()
sp.check_call(['patch', '-p1', '-i',
os.path.join(ci['scriptsdir'], 'add-RTEMS-pc368-qemu-target.patch')],
cwd=places['EPICS_BASE'])
with open(os.path.join(places['EPICS_BASE'], 'configure', 'CONFIG_SITE'), 'a') as f: with open(os.path.join(places['EPICS_BASE'], 'configure', 'CONFIG_SITE'), 'a') as f:
f.write(''' f.write('''
CROSS_COMPILER_TARGET_ARCHS += RTEMS-pc386{0}'''.format(qemu_suffix)) CROSS_COMPILER_TARGET_ARCHS += RTEMS-pc386-qemu''')
host_ccmplr_name = re.sub(r'^([a-zA-Z][^-]*(-[a-zA-Z][^-]*)*)+(-[0-9.]|)$', r'\1', ci['compiler']) host_ccmplr_name = re.sub(r'^([a-zA-Z][^-]*(-[a-zA-Z][^-]*)*)+(-[0-9.]|)$', r'\1', ci['compiler'])
host_cmplr_ver_suffix = re.sub(r'^([a-zA-Z][^-]*(-[a-zA-Z][^-]*)*)+(-[0-9.]|)$', r'\3', ci['compiler']) host_cmplr_ver_suffix = re.sub(r'^([a-zA-Z][^-]*(-[a-zA-Z][^-]*)*)+(-[0-9.]|)$', r'\3', ci['compiler'])
@@ -819,20 +879,24 @@ CMPLR_CLASS = clang''')
CC = {0}{2} CC = {0}{2}
CCC = {1}{2}'''.format(host_ccmplr_name, host_cppcmplr_name, host_cmplr_ver_suffix)) CCC = {1}{2}'''.format(host_ccmplr_name, host_cppcmplr_name, host_cmplr_ver_suffix))
# Add additional flags to CONFIG_SITE # Add additional settings to CONFIG_SITE
flags_text = '' extra_config = ''
if 'USR_CPPFLAGS' in os.environ: if 'USR_CPPFLAGS' in os.environ:
flags_text += ''' extra_config += '''
USR_CPPFLAGS += {0}'''.format(os.environ['USR_CPPFLAGS']) USR_CPPFLAGS += {0}'''.format(os.environ['USR_CPPFLAGS'])
if 'USR_CFLAGS' in os.environ: if 'USR_CFLAGS' in os.environ:
flags_text += ''' extra_config += '''
USR_CFLAGS += {0}'''.format(os.environ['USR_CFLAGS']) USR_CFLAGS += {0}'''.format(os.environ['USR_CFLAGS'])
if 'USR_CXXFLAGS' in os.environ: if 'USR_CXXFLAGS' in os.environ:
flags_text += ''' extra_config += '''
USR_CXXFLAGS += {0}'''.format(os.environ['USR_CXXFLAGS']) USR_CXXFLAGS += {0}'''.format(os.environ['USR_CXXFLAGS'])
if flags_text: if ci['service'] == 'github-actions' and ci['os'] == 'windows':
extra_config += '''
PERL = C:/Strawberry/perl/bin/perl -CSD'''
if extra_config:
with open(os.path.join(places['EPICS_BASE'], 'configure', 'CONFIG_SITE'), 'a') as f: with open(os.path.join(places['EPICS_BASE'], 'configure', 'CONFIG_SITE'), 'a') as f:
f.write(flags_text) f.write(extra_config)
fold_end('set.up.epics_build', 'Configuring EPICS build system') fold_end('set.up.epics_build', 'Configuring EPICS build system')
@@ -841,7 +905,7 @@ USR_CXXFLAGS += {0}'''.format(os.environ['USR_CXXFLAGS'])
if ci['os'] == 'windows' and ci['choco']: if ci['os'] == 'windows' and ci['choco']:
fold_start('install.choco', 'Installing CHOCO packages') fold_start('install.choco', 'Installing CHOCO packages')
sp.check_call(['choco', 'install'] + ci['choco']) sp.check_call(['choco', 'install'] + ci['choco'] + ['-y', '--limitoutput', '--no-progress'])
fold_end('install.choco', 'Installing CHOCO packages') fold_end('install.choco', 'Installing CHOCO packages')
if ci['os'] == 'linux' and ci['apt']: if ci['os'] == 'linux' and ci['apt']:
@@ -849,6 +913,11 @@ USR_CXXFLAGS += {0}'''.format(os.environ['USR_CXXFLAGS'])
sp.check_call(['sudo', 'apt-get', '-y', 'install'] + ci['apt']) sp.check_call(['sudo', 'apt-get', '-y', 'install'] + ci['apt'])
fold_end('install.apt', 'Installing APT packages') fold_end('install.apt', 'Installing APT packages')
if ci['os'] == 'osx' and ci['homebrew']:
fold_start('install.homebrew', 'Installing Homebrew packages')
sp.check_call(['brew', 'install'] + ci['homebrew'])
fold_end('install.homebrew', 'Installing Homebrew packages')
if ci['os'] == 'linux' and 'RTEMS' in os.environ: if ci['os'] == 'linux' and 'RTEMS' in os.environ:
tar_name = 'i386-rtems{0}-trusty-20171203-{0}.tar.bz2'.format(os.environ['RTEMS']) tar_name = 'i386-rtems{0}-trusty-20171203-{0}.tar.bz2'.format(os.environ['RTEMS'])
print('Downloading RTEMS {0} cross compiler: {1}' print('Downloading RTEMS {0} cross compiler: {1}'
@@ -858,7 +927,10 @@ USR_CXXFLAGS += {0}'''.format(os.environ['USR_CXXFLAGS'])
'https://github.com/mdavidsaver/rsb/releases/download/20171203-{0}/{1}' 'https://github.com/mdavidsaver/rsb/releases/download/20171203-{0}/{1}'
.format(os.environ['RTEMS'], tar_name)], .format(os.environ['RTEMS'], tar_name)],
cwd=toolsdir) cwd=toolsdir)
sp.check_call(['tar', '-C', '/', '-xmj', '-f', os.path.join(toolsdir, tar_name)]) sudo_prefix = []
if ci['service'] == 'github-actions':
sudo_prefix = ['sudo']
sp.check_call(sudo_prefix + ['tar', '-C', '/', '-xmj', '-f', os.path.join(toolsdir, tar_name)])
os.remove(os.path.join(toolsdir, tar_name)) os.remove(os.path.join(toolsdir, tar_name))
setup_for_build(args) setup_for_build(args)

57
github-actions/README.md Normal file
View File

@@ -0,0 +1,57 @@
# GitHub Actions Scripts for EPICS Modules
## Features
- 20 parallel runners on Linux/Windows (5 runners on MacOS)
- Ubuntu 16/18/20, MacOS 10.15, Windows Server 2016/2019
- Compile natively on Linux (gcc, clang)
- Compile natively on MacOS (clang)
- Compile natively on Windows (gcc/MinGW, Visual Studio 2017 & 2019)
- Cross-compile for Windows 32bit and 64bit using MinGW and WINE
- Cross-compile for RTEMS 4.9 and 4.10 (Base >= 3.15)
- Caching not supported yet.
## How to Use these Scripts
1. Add the ci-scripts respository as a Git Submodule
(see [README](../README.md) one level above).
2. Add settings files defining which dependencies in which versions
you want to build against
(see [README](../README.md) one level above).
3. Create a GitHub Actions configuration by copying one of the workflow
examples into the directory `.github/workflows` of your module.
```bash
$ mkdir -p .github/workflows
$ cp .ci/github-actions/ci-scripts-build.yml.example-full .github/workflows/ci-scripts-build.yml
```
4. Edit the workflow configuration to include the build jobs you want
GitHub Actions to run.
Build jobs are specified in the `jobs: <job-name>: strategy:`
declaration. The `matrix:` element specifies the axes as configuration
parameters with their lists of values,
`env:` (on the build level) controls the setting of environment variables
(which can be matrix parameters).
The `runs-on:` setting specifies the image (operating system) of the
runner.
The `name:` is what shows up in the web interface for the workflow,
builds and jobs, and the elements under `steps:` describe the actions
executed for each job of the matrix.
Please check the comments in the examples for more hints, and the
[GitHub Actions documentation](https://help.github.com/en/actions)
for a lot more options and details.
5. Push your changes and click on the `Actions` tab of your GitHub repository
page to see your build results.
## Caches
GitHub Actions provides caching of dependencies.
However, since their cache restore and create algorithm is fundamentally
different from those used by Travis and AppVeyor, this will require some
more changes in ci-scripts to work. Be patient.

View File

@@ -0,0 +1,155 @@
# .github/workflows/ci-scripts-build.yml for use with EPICS Base ci-scripts
# (see: https://github.com/epics-base/ci-scripts)
# This is YAML - indentation levels are crucial
# Set the 'name:' properties to values that work for you (MYMODULE)
name: MYMODULE ci-scripts build
# Trigger on pushes and PRs to any branch
on: [push, pull_request]
env:
SETUP_PATH: .ci-local:.ci
SET: test01
CMP: gcc
# For the sequencer on Linux/Windows/MacOS
APT: re2c
CHOCO: re2c
BREW: re2c
jobs:
build-linux:
name: ${{ matrix.cmp }} / ${{ matrix.configuration }} / ${{ matrix.os }}
runs-on: ${{ matrix.os }}
# Set environment variables from matrix parameters
env:
CMP: ${{ matrix.cmp }}
BCFG: ${{ matrix.configuration }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-20.04, ubuntu-18.04, ubuntu-16.04]
cmp: [gcc, clang]
configuration: [default, static, debug, static-debug]
steps:
- uses: actions/checkout@v2
- name: Prepare and compile dependencies
run: python cue.py prepare
- name: Build main module
run: python cue.py build
- name: Run main module tests
run: python cue.py test
- name: Collect and show test results
run: python cue.py test-results
build-macos:
name: ${{ matrix.cmp }} / ${{ matrix.configuration }} / ${{ matrix.os }}
runs-on: ${{ matrix.os }}
# Set environment variables from matrix parameters
env:
CMP: ${{ matrix.cmp }}
BCFG: ${{ matrix.configuration }}
strategy:
fail-fast: false
matrix:
os: [macos-10.15]
cmp: [clang]
# No static builds on MacOS
configuration: [default, debug]
steps:
- uses: actions/checkout@v2
- name: Prepare and compile dependencies
run: python cue.py prepare
- name: Build main module
run: python cue.py build
- name: Run main module tests
run: python cue.py test
- name: Collect and show test results
run: python cue.py test-results
build-windows:
name: ${{ matrix.cmp }} / ${{ matrix.configuration }} / ${{ matrix.os }}
runs-on: ${{ matrix.os }}
env:
CMP: ${{ matrix.cmp }}
BCFG: ${{ matrix.configuration }}
strategy:
fail-fast: false
matrix:
os: [windows-2019, windows-2016]
cmp: [gcc, vs2019, vs2017]
configuration: [default, static, debug, static-debug]
# Available: vs2017/windows-2016 and vs2019/windows-2019
exclude:
- os: windows-2019
cmp: vs2017
- os: windows-2016
cmp: vs2019
steps:
- uses: actions/checkout@v2
- name: Prepare and compile dependencies
run: python cue.py prepare
- name: Build main module
run: python cue.py build
- name: Run main module tests
run: python cue.py test
- name: Collect and show test results
run: python cue.py test-results
# Same setup and toolchain as on Travis.
# Needs Base >= 3.15 to compile, EPICS 7 to also run the tests on qemu
build-rtems:
name: RTEMS${{ matrix.rtems }} / ${{ matrix.configuration }} / ${{ matrix.os }}
runs-on: ${{ matrix.os }}
env:
CMP: ${{ matrix.cmp }}
BCFG: ${{ matrix.configuration }}
RTEMS: ${{ matrix.rtems }}
APT: re2c g++-mingw-w64-i686 g++-mingw-w64-x86-64 qemu-system-x86
strategy:
fail-fast: false
matrix:
os: [ubuntu-18.04]
cmp: [gcc]
configuration: [default, static, debug, static-debug]
rtems: ["4.9", "4.10"]
steps:
- uses: actions/checkout@v2
- name: Prepare and compile dependencies
run: python cue.py prepare
- name: Build main module
run: python cue.py build
- name: Run main module tests
run: python cue.py test
- name: Collect and show test results
run: python cue.py test-results
# The WINE cross builds are of somewhat limited use,
# as there are native gcc/MinGW builds available on GitHub Actions
build-wine:
name: WINE${{ matrix.wine }} / ${{ matrix.configuration }} / ${{ matrix.os }}
runs-on: ${{ matrix.os }}
env:
CMP: ${{ matrix.cmp }}
BCFG: ${{ matrix.configuration }}
WINE: ${{ matrix.wine }}
APT: re2c g++-mingw-w64-i686 g++-mingw-w64-x86-64
strategy:
fail-fast: false
matrix:
os: [ubuntu-18.04]
cmp: [gcc]
configuration: [default, static, debug, static-debug]
wine: [32, 64]
steps:
- uses: actions/checkout@v2
- name: Prepare and compile dependencies
run: python cue.py prepare
- name: Build main module
run: python cue.py build
- name: Run main module tests
run: python cue.py test
- name: Collect and show test results
run: python cue.py test-results

View File

@@ -0,0 +1,43 @@
# .github/workflows/ci-scripts-build.yml for use with EPICS Base ci-scripts
# (see: https://github.com/epics-base/ci-scripts)
# This is YAML - indentation levels are crucial
# Set the 'name:' properties to values that work for you
name: MYMODULE ci-scripts build
# Trigger on pushes and PRs to any branch
on: [push, pull_request]
env:
SETUP_PATH: .ci-local:.ci
SET: test01
CMP: gcc
jobs:
build-linux:
name: ${{ matrix.base }} / ${{ matrix.cmp }} / ${{ matrix.configuration }} / ${{ matrix.os }}
runs-on: ${{ matrix.os }}
# Set environment variables from matrix parameters
env:
CMP: ${{ matrix.cmp }}
BCFG: ${{ matrix.configuration }}
BASE: ${{ matrix.base }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-20.04, ubuntu-18.04]
cmp: [gcc]
configuration: [default, static]
base: ["7.0", "3.15"]
steps:
- uses: actions/checkout@v2
- name: Prepare and compile dependencies
run: python cue.py prepare
- name: Build main module
run: python cue.py build
- name: Run main module tests
run: python cue.py test
- name: Collect and show test results
run: python cue.py test-results

BIN
test.7z Normal file

Binary file not shown.

14
test.patch Normal file
View File

@@ -0,0 +1,14 @@
diff -ruN a/bla.txt b/bla.txt
--- a/bla.txt 2020-06-19 18:54:43.129076711 +0200
+++ b/bla.txt 2020-06-19 18:55:05.093948316 +0200
@@ -1,3 +1,3 @@
-LINE1=YES
+LINE1=NO
LINE2=NO
diff -ruN a/dd/new.txt b/dd/new.txt
--- a/dd/new.txt 1970-01-01 01:00:00.000000000 +0100
+++ b/dd/new.txt 2020-06-19 18:55:35.255032413 +0200
@@ -0,0 +1,2 @@
+NEW LINE 1
+NEW LINE 2

BIN
test.zip Normal file

Binary file not shown.

View File

@@ -31,10 +31,14 @@ addons:
- g++-mingw-w64-x86-64 - g++-mingw-w64-x86-64
# for RTEMS cross builds # for RTEMS cross builds
- qemu-system-x86 - qemu-system-x86
# for .zip/.7z archive hooks
- p7zip-full
homebrew: homebrew:
packages: packages:
# for the sequencer # for the sequencer
- re2c - re2c
# for .zip/.7z archive hooks
- p7zip
update: true update: true
install: install:

View File

@@ -60,3 +60,9 @@ configuration.)
Caches are automatically removed after approx. four weeks. Caches are automatically removed after approx. four weeks.
Your jobs will have to rebuild them once in a while. Your jobs will have to rebuild them once in a while.
## Miscellanea
To use the feature to extract `.zip`/`.7z` archives by setting
`*_HOOK` variables, the Linux and MacOS runners need the APT package
`p7zip-full` resp. the Homebrew package `p7zip` installed.