From aedc10b55d87ee06822af2649cb08773a1821318 Mon Sep 17 00:00:00 2001 From: immarvin Date: Mon, 14 Sep 2015 03:18:04 -0400 Subject: [PATCH 1/4] add doc for accelerating initrd/rootimg generation with pigz --- .../deployment/acc_initrd_rootimg_gen.rst | 60 +++++++++++++++++++ .../acc_initrd_rootimg_gen_ppc64le.rst | 1 + .../diskless/customize_image/index.rst | 3 +- 3 files changed, 63 insertions(+), 1 deletion(-) create mode 100644 docs/source/guides/admin-guides/manage_clusters/common/deployment/acc_initrd_rootimg_gen.rst create mode 100644 docs/source/guides/admin-guides/manage_clusters/ppc64le/diskless/customize_image/acc_initrd_rootimg_gen_ppc64le.rst diff --git a/docs/source/guides/admin-guides/manage_clusters/common/deployment/acc_initrd_rootimg_gen.rst b/docs/source/guides/admin-guides/manage_clusters/common/deployment/acc_initrd_rootimg_gen.rst new file mode 100644 index 000000000..3bfae7859 --- /dev/null +++ b/docs/source/guides/admin-guides/manage_clusters/common/deployment/acc_initrd_rootimg_gen.rst @@ -0,0 +1,60 @@ +Accelerating the diskless initrd and rootimg generating +======================================================== + +Generating diskless initrd with ``genimage`` and compressed rootimg with ``packimage`` and ``liteimg`` is a time-comsuming process, it can be accelerated by enabling paralell compression tool ``pigz`` on the management node with multiple processors and cores. + +The paralell compression tool ``pigz`` can be enabled by installing ``pigz`` package on the management server or diskless rootimg. Depending on the method of generating the initrd and compressed rootimg, the steps differ in different Linux distributions. + + +* Enabling the ``pigz`` on :ref:`Ubuntu Server LTS` +-------------------------------------------------------------------------- + +Make sure the ``pigz`` is installed on the management node with the following command:: + + dpkg -l|grep pigz + +If not, ``pigz`` can be installed with the following command:: + + apt-get install pigz + + +* Enabling the ``pigz`` on :ref:`Suse Linux Enterprise Server (SLES)` +------------------------------------------------------------------------------------------ + +1) Enabling the ``pigz`` in ``genimage`` (only supported in SLES12 or above) + +``pigz`` should be installed in the diskless rootimg, since``pigz`` is shipped in the SLES iso, this can be done by adding ``pigz`` into the ``pkglist`` of diskless osimage. + +2) Enabling the ``pigz`` in ``packimage`` + +Make sure the ``pigz`` is installed on the management node with the following command:: + + rpm -qa|grep pigz + +If not, ``pigz`` can be installed with the following command:: + + zypper install pigz + + +* Enabling the ``pigz`` on :ref:`Red Hat Enterprise Linux (RHEL)` +--------------------------------------------------------------------------------------- + +The package ``pigz`` is shipped in Extra Packages for Enterprise Linux (or EPEL) instead of Redhat iso, this involves some complexity. + +Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special Interest Group that creates, maintains, and manages a high quality set of additional packages for Enterprise Linux, including, but not limited to, Red Hat Enterprise Linux (RHEL), CentOS and Scientific Linux (SL), Oracle Linux (OL). + +EPEL has an ``epel-release`` package that includes gpg keys for package signing and repository information. Installing this package for your Enterprise Linux version should allow you to use normal tools such as ``yum`` to install packages and their dependencies. + +Please refer to the http://fedoraproject.org/wiki/EPEL for more details on EPEL + +1) Enabling the ``pigz`` in ``genimage`` (only supported in RHELS6 or above) + +``pigz`` should be installed in the diskless rootimg. Please download ``pigz`` package from https://dl.fedoraproject.org/pub/epel/ , then customize the diskless osimage to install ``pigz`` as the additional packages, see :doc:`Install Additional Other Packages` for more details. + +2) Enabeling the ``pigz`` in ``packimage`` + +``pigz`` should be installed on the management server. Please download ``pigz`` package from https://dl.fedoraproject.org/pub/epel/ , then install the ``pigz`` with ``yum`` or ``rpm``. + + + + diff --git a/docs/source/guides/admin-guides/manage_clusters/ppc64le/diskless/customize_image/acc_initrd_rootimg_gen_ppc64le.rst b/docs/source/guides/admin-guides/manage_clusters/ppc64le/diskless/customize_image/acc_initrd_rootimg_gen_ppc64le.rst new file mode 100644 index 000000000..f6d7d501c --- /dev/null +++ b/docs/source/guides/admin-guides/manage_clusters/ppc64le/diskless/customize_image/acc_initrd_rootimg_gen_ppc64le.rst @@ -0,0 +1 @@ +.. include:: ../../../common/deployment/acc_initrd_rootimg_gen.rst diff --git a/docs/source/guides/admin-guides/manage_clusters/ppc64le/diskless/customize_image/index.rst b/docs/source/guides/admin-guides/manage_clusters/ppc64le/diskless/customize_image/index.rst index 6e0134d1e..9deae82ba 100644 --- a/docs/source/guides/admin-guides/manage_clusters/ppc64le/diskless/customize_image/index.rst +++ b/docs/source/guides/admin-guides/manage_clusters/ppc64le/diskless/customize_image/index.rst @@ -12,4 +12,5 @@ Optional means all the subitems in this page are not necessary to finish an OS d additional_pkg.rst cfg_second_adapter.rst enable_kdump.rst - install_new_kernel.rst + install_new_kernel.rst + acc_initrd_rootimg_gen_ppc64le.rst From 0ae973b2dcb233cf155e110e43e66b77bcae595f Mon Sep 17 00:00:00 2001 From: immarvin Date: Mon, 14 Sep 2015 09:01:09 -0400 Subject: [PATCH 2/4] add an example on the packimage performance optimized with pigz enabled --- .../deployment/acc_initrd_rootimg_gen.rst | 82 ++++++++++++++++++- 1 file changed, 81 insertions(+), 1 deletion(-) diff --git a/docs/source/guides/admin-guides/manage_clusters/common/deployment/acc_initrd_rootimg_gen.rst b/docs/source/guides/admin-guides/manage_clusters/common/deployment/acc_initrd_rootimg_gen.rst index 3bfae7859..744d76705 100644 --- a/docs/source/guides/admin-guides/manage_clusters/common/deployment/acc_initrd_rootimg_gen.rst +++ b/docs/source/guides/admin-guides/manage_clusters/common/deployment/acc_initrd_rootimg_gen.rst @@ -1,7 +1,7 @@ Accelerating the diskless initrd and rootimg generating ======================================================== -Generating diskless initrd with ``genimage`` and compressed rootimg with ``packimage`` and ``liteimg`` is a time-comsuming process, it can be accelerated by enabling paralell compression tool ``pigz`` on the management node with multiple processors and cores. +Generating diskless initrd with ``genimage`` and compressed rootimg with ``packimage`` and ``liteimg`` is a time-comsuming process, it can be accelerated by enabling paralell compression tool ``pigz`` on the management node with multiple processors and cores. See :ref:`Appendix ` for an example on ``packimage`` performance optimized with ``pigz`` enabled. The paralell compression tool ``pigz`` can be enabled by installing ``pigz`` package on the management server or diskless rootimg. Depending on the method of generating the initrd and compressed rootimg, the steps differ in different Linux distributions. @@ -56,5 +56,85 @@ Please refer to the http://fedoraproject.org/wiki/EPEL for more details on EPEL ``pigz`` should be installed on the management server. Please download ``pigz`` package from https://dl.fedoraproject.org/pub/epel/ , then install the ``pigz`` with ``yum`` or ``rpm``. +.. _pigz_example: + +Appendix: An example on ``packimage`` performance optimization with "pigz" enabled +---------------------------------------------------------------------------------- + +This is an example on performance optimization with ``pigz`` enabled. + +In this example, a xCAT command ``packimage rhels7-ppc64-netboot-compute`` is run on a Power 7 machine with 4 cores. + +The system info: :: + + # uname -a + Linux c910f03c01p03 3.10.0-123.el7.ppc64 #1 SMP Mon May 5 11:18:37 EDT 2014 ppc64 ppc64 ppc64 GNU/Linux + + # cat /etc/os-release + NAME="Red Hat Enterprise Linux Server" + VERSION="7.0 (Maipo)" + ID="rhel" + ID_LIKE="fedora" + VERSION_ID="7.0" + PRETTY_NAME="Red Hat Enterprise Linux Server 7.0 (Maipo)" + ANSI_COLOR="0;31" + CPE_NAME="cpe:/o:redhat:enterprise_linux:7.0:GA:server" + HOME_URL="https://www.redhat.com/" + BUG_REPORT_URL="https://bugzilla.redhat.com/" + + REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7" + REDHAT_BUGZILLA_PRODUCT_VERSION=7.0 + REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" + REDHAT_SUPPORT_PRODUCT_VERSION=7.0 + +The CPU info: :: + + # cat /proc/cpuinfo + processor : 0 + cpu : POWER7 (architected), altivec supported + clock : 3550.000000MHz + revision : 2.0 (pvr 003f 0200) + + processor : 1 + cpu : POWER7 (architected), altivec supported + clock : 3550.000000MHz + revision : 2.0 (pvr 003f 0200) + + processor : 2 + cpu : POWER7 (architected), altivec supported + clock : 3550.000000MHz + revision : 2.0 (pvr 003f 0200) + + processor : 3 + cpu : POWER7 (architected), altivec supported + clock : 3550.000000MHz + revision : 2.0 (pvr 003f 0200) + + timebase : 512000000 + platform : pSeries + model : IBM,8233-E8B + machine : CHRP IBM,8233-E8B + +The time spent on ``packimage`` with ``gzip``: :: + + # time packimage rhels7-ppc64-netboot-compute + Packing contents of /install/netboot/rhels7/ppc64/compute/rootimg + compress method:gzip + + + real 1m14.896s + user 0m0.159s + sys 0m0.019s + +The time spent on ``packimage`` with ``pigz``: :: + + # time packimage rhels7-ppc64-netboot-compute + Packing contents of /install/netboot/rhels7/ppc64/compute/rootimg + compress method:pigz + + real 0m23.177s + user 0m0.176s + sys 0m0.016s + From d7bc545e84a847303357ec3b632cab16f30161a3 Mon Sep 17 00:00:00 2001 From: immarvin Date: Mon, 14 Sep 2015 09:09:52 -0400 Subject: [PATCH 3/4] correct the format --- .../common/deployment/acc_initrd_rootimg_gen.rst | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/source/guides/admin-guides/manage_clusters/common/deployment/acc_initrd_rootimg_gen.rst b/docs/source/guides/admin-guides/manage_clusters/common/deployment/acc_initrd_rootimg_gen.rst index 744d76705..b2abc7e51 100644 --- a/docs/source/guides/admin-guides/manage_clusters/common/deployment/acc_initrd_rootimg_gen.rst +++ b/docs/source/guides/admin-guides/manage_clusters/common/deployment/acc_initrd_rootimg_gen.rst @@ -3,11 +3,15 @@ Accelerating the diskless initrd and rootimg generating Generating diskless initrd with ``genimage`` and compressed rootimg with ``packimage`` and ``liteimg`` is a time-comsuming process, it can be accelerated by enabling paralell compression tool ``pigz`` on the management node with multiple processors and cores. See :ref:`Appendix ` for an example on ``packimage`` performance optimized with ``pigz`` enabled. + + +Enabling the ``pigz`` for diskless initrd and rootimg generating +---------------------------------------------------------------- + The paralell compression tool ``pigz`` can be enabled by installing ``pigz`` package on the management server or diskless rootimg. Depending on the method of generating the initrd and compressed rootimg, the steps differ in different Linux distributions. * Enabling the ``pigz`` on :ref:`Ubuntu Server LTS` --------------------------------------------------------------------------- Make sure the ``pigz`` is installed on the management node with the following command:: @@ -19,7 +23,6 @@ If not, ``pigz`` can be installed with the following command:: * Enabling the ``pigz`` on :ref:`Suse Linux Enterprise Server (SLES)` ------------------------------------------------------------------------------------------- 1) Enabling the ``pigz`` in ``genimage`` (only supported in SLES12 or above) @@ -37,7 +40,6 @@ If not, ``pigz`` can be installed with the following command:: * Enabling the ``pigz`` on :ref:`Red Hat Enterprise Linux (RHEL)` ---------------------------------------------------------------------------------------- The package ``pigz`` is shipped in Extra Packages for Enterprise Linux (or EPEL) instead of Redhat iso, this involves some complexity. From ed15a2f1dda2e3f960972f5c04634a51b6145887 Mon Sep 17 00:00:00 2001 From: immarvin Date: Tue, 15 Sep 2015 02:57:09 -0400 Subject: [PATCH 4/4] correct some indent problems --- .../deployment/acc_initrd_rootimg_gen.rst | 79 +++++++++---------- 1 file changed, 38 insertions(+), 41 deletions(-) diff --git a/docs/source/guides/admin-guides/manage_clusters/common/deployment/acc_initrd_rootimg_gen.rst b/docs/source/guides/admin-guides/manage_clusters/common/deployment/acc_initrd_rootimg_gen.rst index b2abc7e51..c296af68a 100644 --- a/docs/source/guides/admin-guides/manage_clusters/common/deployment/acc_initrd_rootimg_gen.rst +++ b/docs/source/guides/admin-guides/manage_clusters/common/deployment/acc_initrd_rootimg_gen.rst @@ -10,52 +10,49 @@ Enabling the ``pigz`` for diskless initrd and rootimg generating The paralell compression tool ``pigz`` can be enabled by installing ``pigz`` package on the management server or diskless rootimg. Depending on the method of generating the initrd and compressed rootimg, the steps differ in different Linux distributions. +* **[RHEL]** -* Enabling the ``pigz`` on :ref:`Ubuntu Server LTS` + The package ``pigz`` is shipped in Extra Packages for Enterprise Linux (or EPEL) instead of Redhat iso, this involves some complexity. + + Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special Interest Group that creates, maintains, and manages a high quality set of additional packages for Enterprise Linux, including, but not limited to, Red Hat Enterprise Linux (RHEL), CentOS and Scientific Linux (SL), Oracle Linux (OL). + + EPEL has an ``epel-release`` package that includes gpg keys for package signing and repository information. Installing this package for your Enterprise Linux version should allow you to use normal tools such as ``yum`` to install packages and their dependencies. + + Please refer to the http://fedoraproject.org/wiki/EPEL for more details on EPEL + + 1) Enabling the ``pigz`` in ``genimage`` (only supported in RHELS6 or above) + + ``pigz`` should be installed in the diskless rootimg. Please download ``pigz`` package from https://dl.fedoraproject.org/pub/epel/ , then customize the diskless osimage to install ``pigz`` as the additional packages, see :doc:`Install Additional Other Packages` for more details. + + 2) Enabeling the ``pigz`` in ``packimage`` + + ``pigz`` should be installed on the management server. Please download ``pigz`` package from https://dl.fedoraproject.org/pub/epel/ , then install the ``pigz`` with ``yum`` or ``rpm``. -Make sure the ``pigz`` is installed on the management node with the following command:: +* **[UBUNTU]** - dpkg -l|grep pigz + Make sure the ``pigz`` is installed on the management node with the following command:: + + dpkg -l|grep pigz + + If not, ``pigz`` can be installed with the following command:: + + apt-get install pigz -If not, ``pigz`` can be installed with the following command:: - - apt-get install pigz +* **[SLES]** - -* Enabling the ``pigz`` on :ref:`Suse Linux Enterprise Server (SLES)` - -1) Enabling the ``pigz`` in ``genimage`` (only supported in SLES12 or above) - -``pigz`` should be installed in the diskless rootimg, since``pigz`` is shipped in the SLES iso, this can be done by adding ``pigz`` into the ``pkglist`` of diskless osimage. - -2) Enabling the ``pigz`` in ``packimage`` - -Make sure the ``pigz`` is installed on the management node with the following command:: - - rpm -qa|grep pigz - -If not, ``pigz`` can be installed with the following command:: - - zypper install pigz - - -* Enabling the ``pigz`` on :ref:`Red Hat Enterprise Linux (RHEL)` - -The package ``pigz`` is shipped in Extra Packages for Enterprise Linux (or EPEL) instead of Redhat iso, this involves some complexity. - -Extra Packages for Enterprise Linux (or EPEL) is a Fedora Special Interest Group that creates, maintains, and manages a high quality set of additional packages for Enterprise Linux, including, but not limited to, Red Hat Enterprise Linux (RHEL), CentOS and Scientific Linux (SL), Oracle Linux (OL). - -EPEL has an ``epel-release`` package that includes gpg keys for package signing and repository information. Installing this package for your Enterprise Linux version should allow you to use normal tools such as ``yum`` to install packages and their dependencies. - -Please refer to the http://fedoraproject.org/wiki/EPEL for more details on EPEL - -1) Enabling the ``pigz`` in ``genimage`` (only supported in RHELS6 or above) - -``pigz`` should be installed in the diskless rootimg. Please download ``pigz`` package from https://dl.fedoraproject.org/pub/epel/ , then customize the diskless osimage to install ``pigz`` as the additional packages, see :doc:`Install Additional Other Packages` for more details. - -2) Enabeling the ``pigz`` in ``packimage`` - -``pigz`` should be installed on the management server. Please download ``pigz`` package from https://dl.fedoraproject.org/pub/epel/ , then install the ``pigz`` with ``yum`` or ``rpm``. + 1) Enabling the ``pigz`` in ``genimage`` (only supported in SLES12 or above) + + ``pigz`` should be installed in the diskless rootimg, since``pigz`` is shipped in the SLES iso, this can be done by adding ``pigz`` into the ``pkglist`` of diskless osimage. + + 2) Enabling the ``pigz`` in ``packimage`` + + Make sure the ``pigz`` is installed on the management node with the following command:: + + rpm -qa|grep pigz + + If not, ``pigz`` can be installed with the following command:: + + zypper install pigz .. _pigz_example: