Discussion:
Continuous integration and a PR
Otávio Gonçalves de Santana
2018-11-22 13:55:45 UTC
Permalink
Hello everyone, I have a question about PR and continuous integrations.
Why we don't have a plugin to run the tests when we open a PR?

I'm also helping in another Apache Project such as Apache TinkerPop
<https://github.com/apache/tinkerpop/pull/1000> and Apache Commons
<https://github.com/apache/commons-lang/pull/384> Lang and both have this.


I also did a small search about, and I find more projects such as Zeppelin,
Commons-IO, Commons Collection those use the Travis plugin
<https://travis-ci.org>.
Jonathan Gallimore
2018-11-22 14:15:21 UTC
Permalink
We do have a CI server:
https://ci.apache.org/builders/tomee-trunk-ubuntu-jvm8

Our build is massive though - it takes between 2 and 5 hours, depending on
hardware. 6+ PRs being updated per day would mean it would be potentially
running all the time with a growing queue. Commons-lang on the other hand,
takes 15 minutes:
https://travis-ci.org/apache/commons-lang/builds/457567190?utm_source=github_status&utm_medium=notification
.

I'm certainly not against it, but I'm not sure it'll be a trivial setup
either. For those interested, the relevant parts of our buildbot build are
here:

f = util.BuildFactory()
f.addStep(steps.Git(repourl="<git url>/tomee.git", mode='full',
method='clobber', submodules=True, branch="" + scm_path))
f.addStep(steps.ShellCommand(command=["rm", "-rf", tmpdir],
name="remove-tmpdir"))
f.addStep(steps.ShellCommand(command=["mkdir", "-p", tmpdir],
name="create-tmpdir"))
f.addStep(steps.Compile(
command=[
"mvn",
"-e",
"--show-version",
"--update-snapshots",
"--fail-at-end",
"-Djava.io.tmpdir=" + tmpdir,
"clean",
"install",
"-Dhttps.protocols=TLSv1.1,TLSv1.2",
"-DfailIfNoTests=false",
"-DskipTests",
],
env={'MAVEN_OPTS': mopts},
warningPattern="(^WARN) | (^WARNING) | (^warning) |
(.*warning[:].*)",
warnOnWarnings=False,
timeout=2400,
name="compile"
))

f.addStep(steps.ShellCommand(
command=[
"mvn",
"-U",
"-Dhttps.protocols=TLSv1.1,TLSv1.2",
"checkstyle:check",
"-Pstyle"
],
name="check-formatting"
))

f.addStep(steps.ShellCommand(
command=[
"mvn",
"-Dhttps.protocols=TLSv1.1,TLSv1.2",
"pmd:check",
"-Pstyle"
],
name="check-pmd"
))

f.addStep(steps.ShellCommand(command=["rm", "-rf", tmpdir],
name="remove-tmpdir"))
f.addStep(steps.ShellCommand(command=["mkdir", "-p", tmpdir],
name="create-tmpdir"))
f.addStep(steps.Compile(
command=[
"mvn",
"-U",
"--show-version",
"--fail-at-end",
"-Djava.io.tmpdir=" + tmpdir,
"clean",
"install",
"-Pall-adapters",
"-Dmaven.test.error.ignore=true",
"-Dmaven.test.failure.ignore=true",
"-Dhttps.protocols=TLSv1.1,TLSv1.2",
"-Dsurefire.useFile=false",
"-DdisableXmlReport=true",
"-Dopenejb.arquillian.debug=true",
"-DTOMEE_LOCK_FILE=/tmp/" + name + "-tomee.port.lock",
],
env={'MAVEN_OPTS': mopts},
warningPattern="(^WARN) | (^WARNING) | (^warning) |
(.*warning[:].*)",
warnOnWarnings=False,
timeout=18000,
name="test"
))

f.addStep(steps.ShellCommand(
command=["bash", "-c", "find . -name 'org.*.txt' | xargs cat"],
alwaysRun=True,
name="test-results"
))

There's quite a lot going on. Short version:

* checkout
* quick build, no tests
* check style
* PMD
* full build with all tests.

I personally run a server at home and it has a VM just for running builds
before opening PRs. I don't know how much control we get over the build
with Travis or how much power it has. I'm happy to donate some machine
cycles to running builds if we want to try it out.

Jon

On Thu, Nov 22, 2018 at 1:56 PM Otávio Gonçalves de Santana <
Post by Otávio Gonçalves de Santana
Hello everyone, I have a question about PR and continuous integrations.
Why we don't have a plugin to run the tests when we open a PR?
I'm also helping in another Apache Project such as Apache TinkerPop
<https://github.com/apache/tinkerpop/pull/1000> and Apache Commons
<https://github.com/apache/commons-lang/pull/384> Lang and both have this.
I also did a small search about, and I find more projects such as Zeppelin,
Commons-IO, Commons Collection those use the Travis plugin
<https://travis-ci.org>.
Romain Manni-Bucau
2018-11-22 14:18:50 UTC
Permalink
If it helps: Travis is quite bad for big OS projects cause it has a limited
queue for OS project which prevent parallelism very quickly so you would
just wait days at the end, likely better to ask infra to setup the build on
ASF jenkins IMHO. We can have a light flavor of the build, like just a "mvn
clean install -Pall-adapters" - assuming a 3h build still helps.

Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>


Le jeu. 22 nov. 2018 à 15:15, Jonathan Gallimore <
Post by Jonathan Gallimore
https://ci.apache.org/builders/tomee-trunk-ubuntu-jvm8
Our build is massive though - it takes between 2 and 5 hours, depending on
hardware. 6+ PRs being updated per day would mean it would be potentially
running all the time with a growing queue. Commons-lang on the other hand,
https://travis-ci.org/apache/commons-lang/builds/457567190?utm_source=github_status&utm_medium=notification
.
I'm certainly not against it, but I'm not sure it'll be a trivial setup
either. For those interested, the relevant parts of our buildbot build are
f = util.BuildFactory()
f.addStep(steps.Git(repourl="<git url>/tomee.git", mode='full',
method='clobber', submodules=True, branch="" + scm_path))
f.addStep(steps.ShellCommand(command=["rm", "-rf", tmpdir],
name="remove-tmpdir"))
f.addStep(steps.ShellCommand(command=["mkdir", "-p", tmpdir],
name="create-tmpdir"))
f.addStep(steps.Compile(
command=[
"mvn",
"-e",
"--show-version",
"--update-snapshots",
"--fail-at-end",
"-Djava.io.tmpdir=" + tmpdir,
"clean",
"install",
"-Dhttps.protocols=TLSv1.1,TLSv1.2",
"-DfailIfNoTests=false",
"-DskipTests",
],
env={'MAVEN_OPTS': mopts},
warningPattern="(^WARN) | (^WARNING) | (^warning) |
(.*warning[:].*)",
warnOnWarnings=False,
timeout=2400,
name="compile"
))
f.addStep(steps.ShellCommand(
command=[
"mvn",
"-U",
"-Dhttps.protocols=TLSv1.1,TLSv1.2",
"checkstyle:check",
"-Pstyle"
],
name="check-formatting"
))
f.addStep(steps.ShellCommand(
command=[
"mvn",
"-Dhttps.protocols=TLSv1.1,TLSv1.2",
"pmd:check",
"-Pstyle"
],
name="check-pmd"
))
f.addStep(steps.ShellCommand(command=["rm", "-rf", tmpdir],
name="remove-tmpdir"))
f.addStep(steps.ShellCommand(command=["mkdir", "-p", tmpdir],
name="create-tmpdir"))
f.addStep(steps.Compile(
command=[
"mvn",
"-U",
"--show-version",
"--fail-at-end",
"-Djava.io.tmpdir=" + tmpdir,
"clean",
"install",
"-Pall-adapters",
"-Dmaven.test.error.ignore=true",
"-Dmaven.test.failure.ignore=true",
"-Dhttps.protocols=TLSv1.1,TLSv1.2",
"-Dsurefire.useFile=false",
"-DdisableXmlReport=true",
"-Dopenejb.arquillian.debug=true",
"-DTOMEE_LOCK_FILE=/tmp/" + name + "-tomee.port.lock",
],
env={'MAVEN_OPTS': mopts},
warningPattern="(^WARN) | (^WARNING) | (^warning) |
(.*warning[:].*)",
warnOnWarnings=False,
timeout=18000,
name="test"
))
f.addStep(steps.ShellCommand(
command=["bash", "-c", "find . -name 'org.*.txt' | xargs cat"],
alwaysRun=True,
name="test-results"
))
* checkout
* quick build, no tests
* check style
* PMD
* full build with all tests.
I personally run a server at home and it has a VM just for running builds
before opening PRs. I don't know how much control we get over the build
with Travis or how much power it has. I'm happy to donate some machine
cycles to running builds if we want to try it out.
Jon
On Thu, Nov 22, 2018 at 1:56 PM Otávio Gonçalves de Santana <
Post by Otávio Gonçalves de Santana
Hello everyone, I have a question about PR and continuous integrations.
Why we don't have a plugin to run the tests when we open a PR?
I'm also helping in another Apache Project such as Apache TinkerPop
<https://github.com/apache/tinkerpop/pull/1000> and Apache Commons
<https://github.com/apache/commons-lang/pull/384> Lang and both have
this.
Post by Otávio Gonçalves de Santana
I also did a small search about, and I find more projects such as
Zeppelin,
Post by Otávio Gonçalves de Santana
Commons-IO, Commons Collection those use the Travis plugin
<https://travis-ci.org>.
Romain Manni-Bucau
2018-11-22 14:11:06 UTC
Permalink
Hi Otavio,

Just needs to be asked to infra but the reactivity will be ~3h instead of a
few minutes like for small projects so not sure it would help much.
Investing in a build.sh can help more (./build.sh --core, ./build.sh
--tomee, ./build.sh --tomee --validate etc) - it is an old idea nobody
tackled by lack of need at that time

Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>


Le jeu. 22 nov. 2018 à 14:56, Otávio Gonçalves de Santana <
Post by Otávio Gonçalves de Santana
Hello everyone, I have a question about PR and continuous integrations.
Why we don't have a plugin to run the tests when we open a PR?
I'm also helping in another Apache Project such as Apache TinkerPop
<https://github.com/apache/tinkerpop/pull/1000> and Apache Commons
<https://github.com/apache/commons-lang/pull/384> Lang and both have this.
I also did a small search about, and I find more projects such as Zeppelin,
Commons-IO, Commons Collection those use the Travis plugin
<https://travis-ci.org>.
César Hernández Mendoza
2018-11-30 20:12:31 UTC
Permalink
Hi guys,
I need help understanding how I can see past builds results from
https://ci.apache.org/builders/tomee-trunk-ubuntu-jvm8
I couldn't find a way to search for the CI result of one or my PR's I
created early this week.

I would also be interesting to see if we can use Jenkins instead
https://builds.apache.org/
I searched the work Jenkin in https://openejb.markmail.org/search/?q= but
it seems that topic hasn't been often mentioned.
Post by Romain Manni-Bucau
Hi Otavio,
Just needs to be asked to infra but the reactivity will be ~3h instead of a
few minutes like for small projects so not sure it would help much.
Investing in a build.sh can help more (./build.sh --core, ./build.sh
--tomee, ./build.sh --tomee --validate etc) - it is an old idea nobody
tackled by lack of need at that time
Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <
https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<
https://www.packtpub.com/application-development/java-ee-8-high-performance
Le jeu. 22 nov. 2018 à 14:56, Otávio Gonçalves de Santana <
Post by Otávio Gonçalves de Santana
Hello everyone, I have a question about PR and continuous integrations.
Why we don't have a plugin to run the tests when we open a PR?
I'm also helping in another Apache Project such as Apache TinkerPop
<https://github.com/apache/tinkerpop/pull/1000> and Apache Commons
<https://github.com/apache/commons-lang/pull/384> Lang and both have
this.
Post by Otávio Gonçalves de Santana
I also did a small search about, and I find more projects such as
Zeppelin,
Post by Otávio Gonçalves de Santana
Commons-IO, Commons Collection those use the Travis plugin
<https://travis-ci.org>.
--
Atentamente:
César Hernández Mendoza.
Otávio Gonçalves de Santana
2018-12-03 09:36:56 UTC
Permalink
Hey Cesar, I have the same issue. Unfortunately no luck yet.

On Fri, Nov 30, 2018 at 9:12 PM César Hernández Mendoza <
Post by César Hernández Mendoza
Hi guys,
I need help understanding how I can see past builds results from
https://ci.apache.org/builders/tomee-trunk-ubuntu-jvm8
I couldn't find a way to search for the CI result of one or my PR's I
created early this week.
I would also be interesting to see if we can use Jenkins instead
https://builds.apache.org/
I searched the work Jenkin in https://openejb.markmail.org/search/?q= but
it seems that topic hasn't been often mentioned.
El jue., 22 nov. 2018 a las 8:19, Romain Manni-Bucau (<
Post by Romain Manni-Bucau
Hi Otavio,
Just needs to be asked to infra but the reactivity will be ~3h instead
of a
Post by Romain Manni-Bucau
few minutes like for small projects so not sure it would help much.
Investing in a build.sh can help more (./build.sh --core, ./build.sh
--tomee, ./build.sh --tomee --validate etc) - it is an old idea nobody
tackled by lack of need at that time
Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <
https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<
https://www.packtpub.com/application-development/java-ee-8-high-performance
Post by Romain Manni-Bucau
Le jeu. 22 nov. 2018 à 14:56, Otávio Gonçalves de Santana <
Post by Otávio Gonçalves de Santana
Hello everyone, I have a question about PR and continuous integrations.
Why we don't have a plugin to run the tests when we open a PR?
I'm also helping in another Apache Project such as Apache TinkerPop
<https://github.com/apache/tinkerpop/pull/1000> and Apache Commons
<https://github.com/apache/commons-lang/pull/384> Lang and both have
this.
Post by Otávio Gonçalves de Santana
I also did a small search about, and I find more projects such as
Zeppelin,
Post by Otávio Gonçalves de Santana
Commons-IO, Commons Collection those use the Travis plugin
<https://travis-ci.org>.
--
César Hernández Mendoza.
Daniel Cunha
2018-12-03 09:45:17 UTC
Permalink
Hey,

I believe for TomEE we don't have PR integrated with CI. BTW, I also prefer
to have a jenkins instead of build-bot, for sure I think Jenkins better
than build-bot.

Em seg, 3 de dez de 2018 às 06:37, Otávio Gonçalves de Santana <
Post by Otávio Gonçalves de Santana
Hey Cesar, I have the same issue. Unfortunately no luck yet.
On Fri, Nov 30, 2018 at 9:12 PM César Hernández Mendoza <
Post by César Hernández Mendoza
Hi guys,
I need help understanding how I can see past builds results from
https://ci.apache.org/builders/tomee-trunk-ubuntu-jvm8
I couldn't find a way to search for the CI result of one or my PR's I
created early this week.
I would also be interesting to see if we can use Jenkins instead
https://builds.apache.org/
I searched the work Jenkin in https://openejb.markmail.org/search/?q=
but
Post by César Hernández Mendoza
it seems that topic hasn't been often mentioned.
El jue., 22 nov. 2018 a las 8:19, Romain Manni-Bucau (<
Post by Romain Manni-Bucau
Hi Otavio,
Just needs to be asked to infra but the reactivity will be ~3h instead
of a
Post by Romain Manni-Bucau
few minutes like for small projects so not sure it would help much.
Investing in a build.sh can help more (./build.sh --core, ./build.sh
--tomee, ./build.sh --tomee --validate etc) - it is an old idea nobody
tackled by lack of need at that time
Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <
https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<
https://www.packtpub.com/application-development/java-ee-8-high-performance
Post by César Hernández Mendoza
Post by Romain Manni-Bucau
Le jeu. 22 nov. 2018 à 14:56, Otávio Gonçalves de Santana <
Post by Otávio Gonçalves de Santana
Hello everyone, I have a question about PR and continuous
integrations.
Post by César Hernández Mendoza
Post by Romain Manni-Bucau
Post by Otávio Gonçalves de Santana
Why we don't have a plugin to run the tests when we open a PR?
I'm also helping in another Apache Project such as Apache TinkerPop
<https://github.com/apache/tinkerpop/pull/1000> and Apache Commons
<https://github.com/apache/commons-lang/pull/384> Lang and both have
this.
Post by Otávio Gonçalves de Santana
I also did a small search about, and I find more projects such as
Zeppelin,
Post by Otávio Gonçalves de Santana
Commons-IO, Commons Collection those use the Travis plugin
<https://travis-ci.org>.
--
César Hernández Mendoza.
--
Daniel "soro" Cunha
https://twitter.com/dvlc_
Loading...