build.webkit.org should have a clean build button
https://bugs.webkit.org/show_bug.cgi?id=123559
Reviewed by Darin Adler.
Add a new "clean" forced scheduler. When a build was created by this scheduler, CleanBuildIfScheduled
step executes clean-build script that removes the WebKitBuild directory.
Also added make-passwords-json.py which creates a fake passwords.json from config.json for testing purposes.
* BuildSlaveSupport/build.webkit.org-config/make-passwords-json.py: Added.
* BuildSlaveSupport/build.webkit.org-config/master.cfg:
(CleanBuildIfScheduled): Added.
(CleanBuildIfScheduled.start): Added. Don't do a clean build if if this build wasn't schecueld by
the clean build scheduler.
(DeleteStaleBuildFiles): Added.
(DeleteStaleBuildFiles.start): Added. Don't delete stale build files if this build was scheduled by
the clean build scheduler since everything in the WebKitBuild directory had already been removed.
(Factory.__init__): Always add CleanBuildIfScheduled. The step itself checks whether to run
clean-build script or hide itself.
(loadBuilderConfig): Added the new clean build scheduler. Also make ForceScheduler future proof; in newer
versions of buildbot, builderNames needs to be a list of regular strings.
* BuildSlaveSupport/clean-build: Added.
git-svn-id: http://svn.webkit.org/repository/webkit/trunk@158393 268f45cc-cd09-0410-ab3c-d52691b4dbfc
diff --git a/Tools/BuildSlaveSupport/build.webkit.org-config/make-passwords-json.py b/Tools/BuildSlaveSupport/build.webkit.org-config/make-passwords-json.py
new file mode 100755
index 0000000..ff4d67d
--- /dev/null
+++ b/Tools/BuildSlaveSupport/build.webkit.org-config/make-passwords-json.py
@@ -0,0 +1,12 @@
+#!/usr/bin/python
+
+import json
+
+with open('config.json', 'r') as config_file:
+ config = json.load(config_file)
+ passwords = {}
+ for slave in config['slaves']:
+ passwords[slave['name']] = 'a'
+
+with open('passwords.json', 'w') as passwords_file:
+ passwords_file.write(json.dumps(passwords))
diff --git a/Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg b/Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg
index 8a6b4a9..27b1189 100644
--- a/Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg
+++ b/Tools/BuildSlaveSupport/build.webkit.org-config/master.cfg
@@ -4,7 +4,7 @@
from buildbot.buildslave import BuildSlave
from buildbot.changes.pb import PBChangeSource
from buildbot.scheduler import AnyBranchScheduler, Triggerable
-from buildbot.schedulers.forcesched import FixedParameter, ForceScheduler, StringParameter
+from buildbot.schedulers.forcesched import FixedParameter, ForceScheduler, StringParameter, BooleanParameter
from buildbot.schedulers.filter import ChangeFilter
from buildbot.status import html
from buildbot.status.web.authz import Authz
@@ -145,12 +145,30 @@
descriptionDone = ["killed old processes"]
command = ["python", "./Tools/BuildSlaveSupport/kill-old-processes"]
+class CleanBuildIfScheduled(shell.Compile):
+ name = "delete WebKitBuild directory"
+ description = ["deleting WebKitBuild directory"]
+ descriptionDone = ["deleted WebKitBuild directory"]
+ command = ["python", "./Tools/BuildSlaveSupport/clean-build", WithProperties("--platform=%(fullPlatform)s"), WithProperties("--%(configuration)s")]
+
+ def start(self):
+ if not self.getProperty('is_clean'):
+ self.hideStepIf = True
+ return SKIPPED
+ return shell.Compile.start(self)
+
class DeleteStaleBuildFiles(shell.Compile):
name = "delete stale build files"
description = ["deleting stale build files"]
- descriptionDone = ["delete stale build files"]
+ descriptionDone = ["deleted stale build files"]
command = ["python", "./Tools/BuildSlaveSupport/delete-stale-build-files", WithProperties("--platform=%(fullPlatform)s"), WithProperties("--%(configuration)s")]
+ def start(self):
+ if self.getProperty('is_clean'): # Nothing to be done if WebKitBuild had been removed.
+ self.hideStepIf = True
+ return SKIPPED
+ return shell.Compile.start(self)
+
class InstallEflDependencies(shell.ShellCommand):
name = "jhbuild"
description = ["updating efl dependencies"]
@@ -717,6 +735,7 @@
self.addStep(WaitForSVNServer())
self.addStep(CheckOutSource(SVNMirror=SVNMirror))
self.addStep(KillOldProcesses())
+ self.addStep(CleanBuildIfScheduled())
self.addStep(DeleteStaleBuildFiles())
if platform == "win":
self.addStep(InstallWin32Dependencies())
@@ -876,17 +895,17 @@
forceScheduler = ForceScheduler(
name="force",
- builderNames=[builder['name'] for builder in config['builders']],
+ builderNames=[str(builder['name']) for builder in config['builders']],
reason=StringParameter(name="reason", default="", size=40),
- # Validate SVN revision: number or emtpy string
+ # Validate SVN revision: number or empty string
revision=StringParameter(name="revision", default="", regex=re.compile(r'^(\d*)$')),
# Disable default enabled input fields: branch, repository, project, additional properties
branch=FixedParameter(name="branch"),
repository=FixedParameter(name="repository"),
project=FixedParameter(name="project"),
- properties=[]
+ properties=[BooleanParameter(name="is_clean", label="Force Clean build")]
)
c['schedulers'].append(forceScheduler)
diff --git a/Tools/BuildSlaveSupport/clean-build b/Tools/BuildSlaveSupport/clean-build
new file mode 100755
index 0000000..eaecb12
--- /dev/null
+++ b/Tools/BuildSlaveSupport/clean-build
@@ -0,0 +1,54 @@
+#!/usr/bin/python
+# Copyright (C) 2013 Apple Inc. All rights reserved.
+# Copyright (C) 2012 Google Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# 1. Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE AND ITS CONTRIBUTORS "AS IS" AND ANY
+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+# DISCLAIMED. IN NO EVENT SHALL APPLE OR ITS CONTRIBUTORS BE LIABLE FOR ANY
+# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+import optparse
+import os
+import shutil
+import subprocess
+import sys
+
+def main():
+ parser = optparse.OptionParser("usage: %prog [options]")
+ parser.add_option("--platform")
+ parser.add_option("--debug", action="store_const", const="debug", dest="configuration")
+ parser.add_option("--release", action="store_const", const="release", dest="configuration")
+
+ options, parameters = parser.parse_args()
+ if not options.platform:
+ parser.error("Platform is required")
+ return -1
+ if not options.configuration:
+ parser.error("Configuration is required")
+ return -2
+
+ generic_platform = options.platform.split('-', 1)[0]
+
+ webkit_build_directory = subprocess.Popen(['perl', os.path.join(os.path.dirname(__file__), "..", "Scripts", "webkit-build-directory"),
+ "--" + generic_platform, "--" + options.configuration, '--top-level'], stdout=subprocess.PIPE).communicate()[0].strip()
+
+ shutil.rmtree(webkit_build_directory)
+
+if __name__ == '__main__':
+ sys.exit(main())
diff --git a/Tools/ChangeLog b/Tools/ChangeLog
index 9a759a3..e8f00b6 100644
--- a/Tools/ChangeLog
+++ b/Tools/ChangeLog
@@ -1,3 +1,29 @@
+2013-10-30 Ryosuke Niwa <rniwa@webkit.org>
+
+ build.webkit.org should have a clean build button
+ https://bugs.webkit.org/show_bug.cgi?id=123559
+
+ Reviewed by Darin Adler.
+
+ Add a new "clean" forced scheduler. When a build was created by this scheduler, CleanBuildIfScheduled
+ step executes clean-build script that removes the WebKitBuild directory.
+
+ Also added make-passwords-json.py which creates a fake passwords.json from config.json for testing purposes.
+
+ * BuildSlaveSupport/build.webkit.org-config/make-passwords-json.py: Added.
+ * BuildSlaveSupport/build.webkit.org-config/master.cfg:
+ (CleanBuildIfScheduled): Added.
+ (CleanBuildIfScheduled.start): Added. Don't do a clean build if if this build wasn't schecueld by
+ the clean build scheduler.
+ (DeleteStaleBuildFiles): Added.
+ (DeleteStaleBuildFiles.start): Added. Don't delete stale build files if this build was scheduled by
+ the clean build scheduler since everything in the WebKitBuild directory had already been removed.
+ (Factory.__init__): Always add CleanBuildIfScheduled. The step itself checks whether to run
+ clean-build script or hide itself.
+ (loadBuilderConfig): Added the new clean build scheduler. Also make ForceScheduler future proof; in newer
+ versions of buildbot, builderNames needs to be a list of regular strings.
+ * BuildSlaveSupport/clean-build: Added.
+
2013-10-31 Filip Pizlo <fpizlo@apple.com>
Remove CachedTranscendentalFunction because caching math functions is an ugly idea