2
0
mirror of https://github.com/xcat2/confluent.git synced 2026-09-29 08:41:00 +00:00

Compare commits

..

2 Commits

Author SHA1 Message Date
Jarrod Johnson 20457bbe37 Fix issues with leftover ssh sessions
Upon connection loss, even though confluent internally
decides it is done with it, it fails to close the session.

Catch a number of these scenarios and ensure the connection closes.
2020-11-03 09:14:26 -05:00
Jarrod Johnson 206a79a976 Fix certificate watch hang
If no certificate present, then once a day confluent could hang. Fix by
doing a non-blocking
read on the watcher.
2020-11-03 09:07:09 -05:00
585 changed files with 2417 additions and 50845 deletions
-3
View File
@@ -1,7 +1,4 @@
*.pyc
.*.
confluent_client/man/man*
.vscode
.*.sw*
.sw*
.idea/*
+25
View File
@@ -175,3 +175,28 @@
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-30
View File
@@ -1,30 +0,0 @@
# Confluent
![Python 3](https://img.shields.io/badge/python-3-blue.svg) [![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/xcat2/confluent/blob/master/LICENSE)
Confluent is a software package to handle essential bootstrap and operation of scale-out server configurations.
It supports stateful and stateless deployments for various operating systems.
Check [this page](https://hpc.lenovo.com/users/documentation/whatisconfluent.html
) for a more detailed list of features.
Confluent is the modern successor of [xCAT](https://github.com/xcat2/xcat-core).
If you're coming from xCAT, check out [this comparison](https://hpc.lenovo.com/users/documentation/confluentvxcat.html).
# Documentation
Confluent documentation is hosted on hpc.lenovo.com: https://hpc.lenovo.com/users/documentation/
# Download
Get the latest version from: https://hpc.lenovo.com/users/downloads/
Check release notes on: https://hpc.lenovo.com/users/news/
# Open Source License
Confluent is made available under the Apache 2.0 license: https://opensource.org/license/apache-2-0
# Developers
Want to help? Submit a [Pull Request](https://github.com/xcat2/confluent/pulls).
-77
View File
@@ -1,77 +0,0 @@
Project: https://hpc.lenovo.com/users/
Source: https://github.com/lenovo/confluent
Upstream-Name: confluent-client
All file of the Confluent-client software is distributed under the terms of an Apache-2.0 license as indicated below:
Files: *
Copyright: 2014-2019 Lenovo
License: Apache-2.0
File: client.py:
Copyright: 2014 IBM Corporation
2015-2019 Lenovo
License: Apache-2.0
File: sortutil.py,
tlvdata.py
Copyright: 2014 IBM Corporation
2015-2016 Lenovo
License: Apache-2.0
File: tlv.py
Copyright: 2014 IBM Corporation
License: Apache-2.0
License: Apache-2.0
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files.
"Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions:
You must give any other recipients of the Work or Derivative Works a copy of this License; and
You must cause any modified files to carry prominent notices stating that You changed the files; and
You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and
If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License.
You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
+6 -11
View File
@@ -14,16 +14,11 @@ import shutil
shutil.copyfile('doc/man/nodeattrib.ronn.tmpl', 'doc/man/nodeattrib.ronn')
shutil.copyfile('doc/man/nodegroupattrib.ronn.tmpl', 'doc/man/nodegroupattrib.ronn')
with open('doc/man/nodeattrib.ronn', 'a') as outf:
for field in sorted(attr.node):
outf.write('\n* `{0}`:\n {1}\n'.format(field, attr.node[field]['description']))
with open('doc/man/nodegroupattrib.ronn', 'a') as outf:
for field in sorted(attr.node):
outf.write('\n* `{0}`:\n {1}\n'.format(field, attr.node[field]['description']))
def append_attributes(filename):
with open(filename, 'a') as outf:
for field in sorted(attr.node):
outf.write('\n* `{0}`:\n {1}\n'.format(field, attr.node[field]['description']))
# Optionally write valid values if they exist
for key, values in attr.node[field].items():
if key.startswith('valid'):
values_formatted = ', '.join("'{0}'".format(v) for v in values)
outf.write(f'\n Valid values: {values_formatted}\n')
append_attributes('doc/man/nodeattrib.ronn')
append_attributes('doc/man/nodegroupattrib.ronn')
+3 -13
View File
@@ -21,7 +21,6 @@
import optparse
import os
import re
import select
import sys
@@ -85,28 +84,19 @@ fullline = sys.stdin.readline()
printpending = True
clearpending = False
holdoff = 0
padded = None
while fullline:
for line in fullline.split('\n'):
if not line:
continue
if ':' not in line:
if ': ' not in line:
line = 'UNKNOWN: ' + line
if options.log:
node, output = line.split(':', 1)
if padded is None:
if output.startswith(' '):
padded = True
else:
padded = False
if padded:
output = re.sub(r'^ ', '', output)
node, output = line.split(': ', 1)
currlog = options.log.format(node=node, nodename=node)
with open(currlog, mode='a') as log:
log.write(output + '\n')
continue
node, output = line.split(':', 1)
grouped.add_line(node, output)
grouped.add_line(*line.split(': ', 1))
if options.watch:
if not holdoff:
holdoff = os.times()[4] + 0.250
+98 -130
View File
@@ -1,4 +1,4 @@
#!/usr/bin/python3
#!/usr/bin/python2
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2014 IBM Corporation
@@ -292,8 +292,6 @@ def rcompleter(text, state):
def parse_command(command):
if isinstance(command, list):
return command
try:
args = shlex.split(command, posix=True)
except ValueError as ve:
@@ -373,7 +371,7 @@ def do_command(command, server):
if argv[0] == 'exit':
if os.environ.get('TERM', '') not in ('linux'):
sys.stdout.write('\x1b]0;\x07')
raise BailOut()
raise Bailout()
elif argv[0] in ('help', '?'):
return print_help()
elif argv[0] == 'cd':
@@ -635,7 +633,7 @@ def quitconfetty(code=0, fullexit=False, fixterm=True):
global didconsole
if fixterm or didconsole:
currfl = fcntl.fcntl(sys.stdin.fileno(), fcntl.F_GETFL)
fcntl.fcntl(sys.stdin.fileno(), fcntl.F_SETFL, currfl & ~os.O_NONBLOCK)
fcntl.fcntl(sys.stdin.fileno(), fcntl.F_SETFL, currfl ^ os.O_NONBLOCK)
if oldtcattr is not None:
termios.tcsetattr(sys.stdin.fileno(), termios.TCSANOW, oldtcattr)
# Request default color scheme, to undo potential weirdness of terminal
@@ -654,41 +652,15 @@ def quitconfetty(code=0, fullexit=False, fixterm=True):
def get_session_node(shellargs):
# straight to node console
if len(shellargs) == 1 and ' ' not in shellargs[0]:
targ = "/nodes/%s/console/session" % shellargs[0]
return targ, shellargs[0]
return shellargs[0]
if len(shellargs) == 2 and shellargs[0] == 'start':
args = [s for s in shellargs[1].split('/') if s]
if len(args) == 4 and args[0] == 'nodes' and args[2] == 'console' and \
args[3] == 'session':
return shellargs[1], args[1]
if len(args) == 5 and args[0] == 'nodes' and args[2] == 'shell' and \
args[3] == 'sessions':
return shellargs[1], args[1]
return None, None
return args[1]
return None
def run_inline_command(path, arg, completion, **kwargs):
tlvdata.send(session.connection, {'operation': 'pause',
'path': currconsole})
buffdata = ''
while select.select((session.connection,), (), (), 0)[0]:
buffdata += consume_termdata(session.connection, bufferonly=True)
rc = session.simple_noderange_command(consolename, path, arg, **kwargs)
tlvdata.send(session.connection, {'operation': 'resume',
'path': currconsole})
sys.stdout.write(completion)
sys.stdout.flush()
if buffdata:
try:
sys.stdout.write(buffdata)
except UnicodeEncodeError:
sys.stdout.buffer.write(buffdata.encode('utf8'))
except IOError: # Some times circumstances are bad
# resort to byte at a time...
for d in buffdata:
sys.stdout.write(d)
return rc
def conserver_command(filehandle, localcommand):
# x - conserver has that as 'show baud', I am inclined to replace that with
# 'request exclusive'
@@ -724,13 +696,13 @@ def conserver_command(filehandle, localcommand):
localcommand = get_command_bytes(filehandle, localcommand, cmdlen)
if localcommand[1] == 'o': # off
sys.stdout.write("powering off...")
sys.stdout.flush()
run_inline_command('/power/state', 'off', 'complete]')
print("powering off...")
session.simple_noderange_command(consolename, '/power/state', 'off')
print("complete]\r")
elif localcommand[1] == 's': # shutdown
sys.stdout.write("shutting down...")
sys.stdout.flush()
run_inline_command('/power/state', 'shutdown', 'complete]')
print("shutting down...")
session.simple_noderange_command(consolename, '/power/state', 'shutdown')
print("complete]\r")
elif localcommand[1] == 'b': # boot
cmdlen += 1
localcommand = get_command_bytes(filehandle, localcommand, cmdlen)
@@ -741,46 +713,59 @@ def conserver_command(filehandle, localcommand):
bootmode = 'uefi'
bootdev = 'setup'
rc = run_inline_command('/boot/nextdevice', bootdev, '', bootmode=bootmode)
rc = session.simple_noderange_command(consolename, '/boot/nextdevice', bootdev, bootmode=bootmode)
if rc:
print("Error]\r")
else:
rc = run_inline_command('/power/state', 'boot', 'complete]')
if rc:
print("Error]")
elif localcommand[2] == 'n': # boot to network
sys.stdout.write("booting to network...")
sys.stdout.flush()
bootmode = 'uefi'
bootdev = 'network'
rc = run_inline_command('/boot/nextdevice', bootdev, '', bootmode=bootmode)
if rc:
print("Error]\r")
else:
rc = run_inline_command('/power/state', 'boot', 'complete]')
rc = session.simple_noderange_command(consolename, '/power/state', 'boot')
if rc:
print("Error]\r")
else:
print("complete]\r")
elif localcommand[2] == 'n': # boot to network
print("booting to network...")
bootmode = 'uefi'
bootdev = 'network'
rc = session.simple_noderange_command(consolename, '/boot/nextdevice', bootdev, bootmode=bootmode)
if rc:
print("Error]\r")
else:
rc = session.simple_noderange_command(consolename, '/power/state', 'boot')
if rc:
print("Error]\r")
else:
print("complete]\r")
elif localcommand[2] == '\x0d': # boot to default
print("booting to default...")
bootmode = 'uefi'
bootdev = 'default'
rc = run_inline_command('/boot/nextdevice', bootdev, '', bootmode=bootmode)
rc = session.simple_noderange_command(consolename, '/boot/nextdevice', bootdev, bootmode=bootmode)
if rc:
print("Error]\r")
else:
rc = run_inline_command('/power/state', 'boot', 'complete]')
rc = session.simple_noderange_command(consolename, '/power/state', 'boot')
if rc:
print("Error]\r")
else:
print("complete]\r")
else:
print("Unknown boot state.]\r")
else:
print("Unknown power state.]\r")
check_power_state()
elif localcommand[0] == 'r':
sys.stdout.write('\x1b7\x1b[999;999H\x1b[6n')
@@ -921,13 +906,14 @@ def main():
doexit = False
inconsole = False
pendingcommand = ""
targ, session_node = get_session_node(shellargs)
session_node = get_session_node(shellargs)
if session_node is not None:
consoleonly = True
do_command("start %s" % targ, netserver)
do_command("start /nodes/%s/console/session" % session_node, netserver)
doexit = True
elif shellargs:
do_command(shellargs, netserver)
command = " ".join(shellargs)
do_command(command, netserver)
quitconfetty(fullexit=True, fixterm=False)
powerstate = None
@@ -942,7 +928,57 @@ def main():
rdylist = ()
for fh in rdylist:
if fh == session.connection:
consume_termdata(fh)
# this only should get called in the
# case of a console session
# each command should slurp up all relevant
# recv potential
#fh.read()
try:
data = tlvdata.recv(fh)
except Exception:
data = None
if type(data) == dict:
updatestatus(data)
continue
if data is not None:
data = client.stringify(data)
if clearpowermessage:
sys.stdout.write("\x1b[2J\x1b[;H")
clearpowermessage = False
try:
sys.stdout.write(data)
except IOError: # Some times circumstances are bad
# resort to byte at a time...
for d in data:
sys.stdout.write(d)
now = time.time()
if ('showtime' not in laststate or
(now // 60) != laststate['showtime'] // 60):
# don't bother churning if minute does not change
laststate['showtime'] = now
updatestatus()
sys.stdout.flush()
else:
deadline = 5
connected = False
while not connected and deadline > 0:
try:
server_connect()
connected = True
except Exception:
pass
if not connected:
time.sleep(1)
deadline -=1
if connected:
do_command(
"start /nodes/%s/console/session skipreplay=True" % consolename,
netserver)
else:
doexit = True
inconsole = False
sys.stdout.write("\r\n[remote disconnected]\r\n")
break
else:
try:
myinput = fh.read()
@@ -952,9 +988,8 @@ def main():
except IOError:
pass
if powerstate is None or powertime < time.time() - 10: # Check powerstate every 10 seconds
if powerstate == None:
powerstate = True
powertime = time.time()
powerstate = True
check_power_state()
else:
currcommand = prompt()
@@ -969,65 +1004,6 @@ def main():
sys.stdout.write('Lost connection to server')
quitconfetty(fullexit=True)
def consume_termdata(fh, bufferonly=False):
global clearpowermessage
try:
data = tlvdata.recv(fh)
except Exception:
data = None
if type(data) == dict:
updatestatus(data)
return ''
if data is not None:
data = client.stringify(data)
if clearpowermessage:
sys.stdout.write("\x1b[2J\x1b[;H")
clearpowermessage = False
if bufferonly:
return data
try:
sys.stdout.write(data)
except UnicodeEncodeError:
sys.stdout.buffer.write(data.encode('utf8'))
except IOError: # Some times circumstances are bad
# resort to byte at a time...
for d in data:
sys.stdout.write(d)
now = time.time()
if ('showtime' not in laststate or
(now // 60) != laststate['showtime'] // 60):
# don't bother churning if minute does not change
laststate['showtime'] = now
updatestatus()
try:
sys.stdout.flush()
except Exception:
# EWOULDBLOCK causes this to raise, ignore
# this scenario comfortable that it
# will come out soon enough
pass
else:
deadline = 5
connected = False
while not connected and deadline > 0:
try:
server_connect()
connected = True
except Exception:
pass
if not connected:
time.sleep(1)
deadline -=1
if connected:
do_command(
"start /nodes/%s/console/session skipreplay=True" % consolename,
netserver)
else:
doexit = True
inconsole = False
sys.stdout.write("\r\n[remote disconnected]\r\n")
return ''
if __name__ == '__main__':
errcode = 0
deadline = 0
@@ -1037,14 +1013,6 @@ if __name__ == '__main__':
main()
except BailOut as e:
errcode = e.errorcode
except Exception as e:
import traceback
excinfo = traceback.print_exc()
try:
quitconfetty()
except Exception:
pass
print(excinfo)
finally:
if deadline and os.times()[4] < deadline:
sys.stderr.write('[Exited early, hit enter to continue]')
-219
View File
@@ -1,219 +0,0 @@
#!/usr/bin/python2
import argparse
import os
import re
import signal
import sys
try:
signal.signal(signal.SIGPIPE, signal.SIG_DFL)
except AttributeError:
pass
path = os.path.dirname(os.path.realpath(__file__))
path = os.path.realpath(os.path.join(path, '..', 'lib', 'python'))
if path.startswith('/opt'):
sys.path.append(path)
import confluent.client as client
import confluent.sortutil as sortutil
def partitionhostsline(line):
comment = ''
if '#' in line:
cmdidx = line.index('#')
comment = line[cmdidx:]
line = line[:cmdidx].strip()
if not line.strip():
return '', [], comment
ipaddr, names = line.split(None, 1)
names = names.split()
return ipaddr, names, comment
class HostMerger(object):
def __init__(self):
self.byip = {}
self.byname = {}
self.byname6 = {}
self.sourcelines = []
self.targlines = []
def read_source(self, sourcefile):
with open(sourcefile, 'r') as hfile:
self.sourcelines = hfile.read().split('\n')
while not self.sourcelines[-1]:
self.sourcelines = self.sourcelines[:-1]
for x in range(len(self.sourcelines)):
line = self.sourcelines[x]
currip, names, comment = partitionhostsline(line)
if currip:
self.byip[currip] = x
byname = self.byname
if ':' in currip:
byname = self.byname6
for name in names:
byname[name] = x
def add_entry(self, ip, names):
targ = self.byname
if ':' in ip:
targ = self.byname6
line = '{:<39} {}'.format(ip, names)
x = len(self.sourcelines)
self.sourcelines.append(line)
for name in names.split():
if not name:
continue
targ[name] = x
self.byip[ip] = x
def read_target(self, targetfile):
if not os.path.exists(targetfile):
return
with open(targetfile, 'r') as hfile:
lines = hfile.read().split('\n')
while lines and not lines[-1]:
lines = lines[:-1]
for y in range(len(lines)):
line = lines[y]
currip, names, comment = partitionhostsline(line)
byname = self.byname
if ':' in currip:
byname = self.byname6
if currip in self.byip:
x = self.byip[currip]
if self.sourcelines[x] is None:
# have already consumed this entry
continue
self.targlines.append(self.sourcelines[x])
self.sourcelines[x] = None
continue
for name in names:
if name in byname:
x = byname[name]
if self.sourcelines[x] is None:
break
self.targlines.append(self.sourcelines[x])
self.sourcelines[x] = None
break
else:
self.targlines.append(line)
def write_out(self, targetfile):
while self.targlines and not self.targlines[-1]:
self.targlines = self.targlines[:-1]
while self.sourcelines and not self.sourcelines[-1]:
self.sourcelines = self.sourcelines[:-1]
if not self.sourcelines:
break
with open(targetfile, 'w') as hosts:
for line in self.targlines:
hosts.write(line + '\n')
for line in self.sourcelines:
if line is not None:
hosts.write(line + '\n')
def main():
ap = argparse.ArgumentParser(description="Create/amend /etc/hosts file for given noderange")
ap.add_argument('noderange', help='Noderange to generate/update /etc/hosts for')
ap.add_argument('-a', '--attrib', help='Pull ip addresses and hostnames from attribute database', action='store_true')
ap.add_argument('-i', '--ip', help='Expression to generate addresses (e.g. 172.16.1.{n1} or fd2b:246f:8a50::{n1:x})')
ap.add_argument('-n', '--name', help='Expression for name to add ({node}-compute, etc). If unspecified, "{node} {node}.{dns.domain}" will be used', action='append')
ap.add_argument('-f', '--fqdn-first', help='Put the FQDN first in the hosts entries', action='store_true')
args = ap.parse_args()
c = client.Command()
if args.name:
names = ' '.join(args.name)
else:
names = '{node} {node}.{dns.domain}'
merger = HostMerger()
if not args.ip and not args.attrib:
sys.stderr.write('-a or -i is currently required\n')
sys.exit(1)
if args.attrib:
ip4bynode = {}
ip6bynode = {}
namesbynode = {}
domainsbynode = {}
for ent in c.read('/noderange/{0}/attributes/current'.format(args.noderange)):
ent = ent.get('databynode', {})
for node in ent:
for attrib in ent[node]:
val = ent[node][attrib]
if not isinstance(val, dict):
continue
val = val.get('value', None)
if attrib == 'dns.domain':
domainsbynode[node] = val
if attrib.startswith('net.'):
nameparts = attrib.split('.')
if len(nameparts) == 2:
currnet = None
else:
currnet = '.'.join(nameparts[1:-1])
for currdict in (ip4bynode, ip6bynode, namesbynode):
if node not in currdict:
currdict[node] = {}
if attrib.endswith('.ipv4_address') and val:
ip4bynode[node][currnet] = val.split('/', 1)[0]
elif attrib.endswith('.ipv6_address') and val:
ip6bynode[node][currnet] = val.split('/', 1)[0]
elif attrib.endswith('.hostname'):
namesbynode[node][currnet] = re.split(r'\s+|,', val)
for node in ip4bynode:
mydomain = domainsbynode.get(node, None)
for ipdb in (ip4bynode, ip6bynode):
for currnet in ipdb[node]:
if args.name:
sys.stderr.write('Custom name and attribute addresses not currently supported together\n')
sys.exit(1)
else:
names = list(namesbynode.get(node, {}).get(currnet, [node]))
if mydomain:
for name in names:
if mydomain in names:
break
else:
for name in list(names):
fqdn = '{0}.{1}'.format(name, mydomain)
if args.fqdn_first:
# Insert FQDN at the beginning if --fqdn-first flag is set
names.insert(0, fqdn)
else:
# Otherwise, append FQDN at the end (original behavior)
names.append(fqdn)
names = ' '.join(names)
merger.add_entry(ipdb[node][currnet], names)
merger.write_out('/etc/whatnowhosts')
else:
namesbynode = {}
ipbynode = {}
expurl = '/noderange/{0}/attributes/expression'.format(args.noderange)
expression = names
exitcode = 0
exitcode |= expand_expression(c, namesbynode, expurl, names)
exitcode |= expand_expression(c, ipbynode, expurl, args.ip)
if exitcode:
sys.exit(exitcode)
for node in ipbynode:
merger.add_entry(ipbynode[node], namesbynode[node])
if os.path.exists('/etc/hosts'):
merger.read_target('/etc/hosts')
os.rename('/etc/hosts', '/etc/hosts.confluentbkup')
merger.write_out('/etc/hosts')
def expand_expression(c, namesbynode, expurl, expression):
exitcode = 0
for exp in c.create(expurl, {'expression': expression}):
if 'error' in exp:
sys.stderr.write(exp['error'] + '\n')
exitcode |= exp.get('errorcode', 1)
ex = exp.get('databynode', ())
for node in ex:
namesbynode[node] = ex[node]['value']
return exitcode
if __name__ == '__main__':
main()
-2
View File
@@ -79,8 +79,6 @@ def collect_attribute_data(databynode, node):
xcatattrs['groups'] = ','.join(data.get('groups', []))
xcatattrs['bmc'] = data.get('hardwaremanagement.manager', {}).get(
'value', None)
if xcatattrs['bmc']:
xcatattrs['bmc'] = xcatattrs['bmc'].split('/', 1)[0]
xcatattrs['mpa'] = data.get('enclosure.manager', {}).get(
'value', None)
xcatattrs['slotid'] = data.get('enclosure.bay', {}).get(
+16 -78
View File
@@ -1,81 +1,19 @@
#!/usr/bin/python
#!/bin/sh
# This will take a given directory and make a 'big floppy image'
# out of it, suitable for nodemedia upload.
import fcntl
import glob
import os
import subprocess
import sys
def create_image(directory, image, label=None, esize=0, totalsize=None):
if totalsize:
datasz = totalsize * 1048576
else:
ents = 0
datasz = 512 + (esize * 1048576)
for dir in os.walk(sys.argv[1]):
ents += 1
for filen in dir[2]:
ents += 1
filename = os.path.join(dir[0], filen)
currsz = os.path.getsize(filename)
# assuming up to 65k cluster
currsz = (currsz // 512 + 1) * 512
datasz += currsz
datasz += ents * 32768
datasz = datasz // 65536 + 1
with open(image, 'wb') as imgfile:
imgfile.seek(datasz * 65536 - 1)
imgfile.write(b'\x00')
if label:
# 4 heads, 32 sectors, means 65k per track
subprocess.check_call(['mformat', '-i', image, '-v', label,
'-r', '16', '-d', '1', '-t', str(datasz),
'-s', '32','-h', '4', '::'])
else:
subprocess.check_call(['mformat', '-i', image, '-r', '16', '-d', '1', '-t',
str(datasz), '-s', '32','-h', '4', '::'])
# Some clustered filesystems will have the lock from mformat
# linger after close (mformat doesn't unlock)
# do a blocking wait for shared lock and then explicitly
# unlock between calls to mtools
with open(image, 'rb') as imgfile:
fcntl.flock(imgfile.fileno(), fcntl.LOCK_SH)
fcntl.flock(imgfile.fileno(), fcntl.LOCK_UN)
cpycmd = ['mcopy', '-i', image, '-s']
cpycmd.extend(glob.glob('{0}/*'.format(directory)))
cpycmd.append('::')
subprocess.check_call(cpycmd)
# though not necessary for us, make sure dir2img doesn't have a lingering
# flock from mcopy for any subsequent commands
with open(image, 'rb') as imgfile:
fcntl.flock(imgfile.fileno(), fcntl.LOCK_SH)
fcntl.flock(imgfile.fileno(), fcntl.LOCK_UN)
if __name__ == '__main__':
if len(sys.argv) < 3:
sys.stderr.write("Usage: {0} <directory> <imagefile>".format(
sys.argv[0]))
sys.exit(1)
label = None
args = sys.argv
esize = 0
try:
earg = args.index('-e')
esize = int(args[earg + 1])
args = args[:earg] + args[earg +2:]
except ValueError:
pass
totsize = None
try:
earg = args.index('-s')
totsize = int(args[earg + 1])
args = args[:earg] + args[earg +2:]
except ValueError:
pass
if len(args) > 3:
label = args[3]
create_image(args[1], args[2], label, esize, totsize)
if [ -z "$1" -o -z "$2" ]; then
echo "Usage: $0 <directory> <imagefile>"
exit 1
fi
# Get the needed payload side
SIZE=$(du -sB 512 $1|awk '{print $1}')
ENTRIES=$(find $1 |wc -l)
# Also, each file and directory has overhead, pad size by 32kb per file,
# which should be overkill but other values proved to be inadequate
# A deeper understanding of VFAT would probably allow for more precise value
SIZE=$((SIZE + ENTRIES * 64))
dd if=/dev/zero of=$2 bs=512 count=$SIZE
# Make a big single sided floppy with many many tracks, saves on complex math
mformat -i $2 -d 1 -t $SIZE -s 1 -h 1 ::
mcopy -i $2 -s $1/* ::
-166
View File
@@ -1,166 +0,0 @@
#!/usr/bin/python3
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2016-2017 Lenovo
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from collections import deque
import optparse
import os
import select
import signal
import subprocess
import sys
try:
signal.signal(signal.SIGPIPE, signal.SIG_DFL)
except AttributeError:
pass
path = os.path.dirname(os.path.realpath(__file__))
path = os.path.realpath(os.path.join(path, '..', 'lib', 'python'))
if path.startswith('/opt'):
sys.path.append(path)
import confluent.client as client
import confluent.sortutil as sortutil
devnull = None
def run():
global devnull
devnull = open(os.devnull, 'rb')
argparser = optparse.OptionParser(
usage="Usage: %prog [options] noderange commandexpression",
epilog="Expressions are the same as in attributes, e.g. "
"'ipmitool -H {hardwaremanagement.manager}' will be expanded.")
argparser.add_option('-f', '-c', '--count', type='int', default=168,
help='Number of commands to run at a time')
argparser.add_option('-k', '--security', action='store_true',
help='Update SSH setup')
argparser.add_option('-F', '--sync', action='store_true',
help='Run the syncfiles associated with the currently completed OS profile on the noderange')
argparser.add_option('-P', '--scripts',
help='Re-run specified scripts, with full path under scripts, e.g. post.d/first,firstboot.d/second')
argparser.add_option('-m', '--maxnodes', type='int',
help='Specify a maximum number of '
'nodes to run remote ssh command to, '
'prompting if over the threshold')
# among other things, FD_SETSIZE limits. Besides, spawning too many
# processes can be unkind for the unaware on memory pressure and such...
#argparser.disable_interspersed_args()
(options, args) = argparser.parse_args()
if len(args) < 1:
argparser.print_help()
sys.exit(1)
client.check_globbing(args[0])
concurrentprocs = options.count
c = client.Command()
currprocs = 0
all = set([])
pipedesc = {}
pendingexecs = deque()
exitcode = 0
c.stop_if_noderange_over(args[0], options.maxnodes)
nodemap = {}
cmdparms = []
nodes = []
for res in c.read('/noderange/{0}/nodes/'.format(args[0])):
if 'error' in res:
sys.stderr.write(res['error'] + '\n')
exitcode |= res.get('errorcode', 1)
break
node = res['item']['href'][:-1]
nodes.append(node)
cmdstorun = []
if options.security:
cmdstorun.append(['run_remote', 'setupssh'])
if options.sync:
cmdstorun.append(['run_remote_python', 'syncfileclient'])
if options.scripts:
for script in options.scripts.split(','):
cmdstorun.append(['run_remote', script])
if not cmdstorun:
argparser.print_help()
sys.exit(1)
idxbynode = {}
cmdvbase = ['bash', '/etc/confluent/functions']
for sshnode in nodes:
idxbynode[sshnode] = 1
cmdv = ['ssh', sshnode] + cmdvbase + cmdstorun[0]
if currprocs < concurrentprocs:
currprocs += 1
run_cmdv(sshnode, cmdv, all, pipedesc)
else:
pendingexecs.append((sshnode, cmdv))
if not all or exitcode:
sys.exit(exitcode)
rdy, _, _ = select.select(all, [], [], 10)
while all:
pernodeout = {}
for r in rdy:
desc = pipedesc[r]
node = desc['node']
data = True
while data and select.select([r], [], [], 0)[0]:
data = r.readline()
if data:
if desc['type'] == 'stdout':
if node not in pernodeout:
pernodeout[node] = []
pernodeout[node].append(data)
else:
data = client.stringify(data)
sys.stderr.write('{0}: {1}'.format(node, data))
sys.stderr.flush()
else:
pop = desc['popen']
ret = pop.poll()
if ret is not None:
exitcode = exitcode | ret
all.discard(r)
r.close()
if desc['type'] == 'stdout':
if idxbynode[node] < len(cmdstorun):
cmdv = ['ssh', sshnode] + cmdvbase + cmdstorun[idxbynode[node]]
idxbynode[node] += 1
run_cmdv(node, cmdv, all, pipedesc)
elif pendingexecs:
node, cmdv = pendingexecs.popleft()
run_cmdv(node, cmdv, all, pipedesc)
for node in sortutil.natural_sort(pernodeout):
for line in pernodeout[node]:
line = client.stringify(line)
line = line.lstrip('\x08')
sys.stdout.write('{0}: {1}'.format(node, line))
sys.stdout.flush()
if all:
rdy, _, _ = select.select(all, [], [], 10)
sys.exit(exitcode)
def run_cmdv(node, cmdv, all, pipedesc):
nopen = subprocess.Popen(
cmdv, stdin=devnull, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
pipedesc[nopen.stdout] = {'node': node, 'popen': nopen,
'type': 'stdout'}
pipedesc[nopen.stderr] = {'node': node, 'popen': nopen,
'type': 'stderr'}
all.add(nopen.stdout)
all.add(nopen.stderr)
if __name__ == '__main__':
run()
-24
View File
@@ -22,7 +22,6 @@ import optparse
import os
import signal
import sys
import shlex
try:
signal.signal(signal.SIGPIPE, signal.SIG_DFL)
@@ -54,8 +53,6 @@ argparser.add_option('-p', '--prompt', action='store_true',
argparser.add_option('-m', '--maxnodes', type='int',
help='Prompt if trying to set attributes on more '
'than specified number of nodes')
argparser.add_option('-s', '--set', dest='set', metavar='settings.batch',
default=False, help='set attributes using a batch file')
(options, args) = argparser.parse_args()
@@ -108,32 +105,11 @@ if len(args) > 1:
requestargs=args[1:]
except:
pass
elif options.clear or options.environment or options.prompt:
sys.stderr.write('Attribute names required with specified options\n')
argparser.print_help()
exitcode = 400
elif options.set:
arglist = [noderange]
showtype='current'
argfile = open(options.set, 'r')
argset = argfile.readline()
while argset:
try:
argset = argset[:argset.index('#')]
except ValueError:
pass
argset = argset.strip()
if argset:
arglist += shlex.split(argset)
argset = argfile.readline()
session.stop_if_noderange_over(noderange, options.maxnodes)
exitcode=client.updateattrib(session,arglist,nodetype, noderange, options, None)
if exitcode != 0:
sys.exit(exitcode)
# Lists all attributes
if len(args) > 0:
# setting output to all so it can search since if we do have something to search, we want to show all outputs even if it is blank.
if requestargs is None:
-121
View File
@@ -1,121 +0,0 @@
#!/usr/libexec/platform-python
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2015-2017 Lenovo
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
__author__ = 'tkucherera'
from getpass import getpass
import optparse
import os
import signal
import sys
try:
signal.signal(signal.SIGPIPE, signal.SIG_DFL)
except AttributeError:
pass
path = os.path.dirname(os.path.realpath(__file__))
path = os.path.realpath(os.path.join(path, '..', 'lib', 'python'))
if path.startswith('/opt'):
sys.path.append(path)
import confluent.client as client
argparser = optparse.OptionParser(usage="Usage: %prog <noderange> <username> <new_password>")
argparser.add_option('-m', '--maxnodes', type='int',
help='Number of nodes to affect before prompting for confirmation')
argparser.add_option('-p', '--prompt', action='store_true',
help='Prompt for password values interactively')
argparser.add_option('-e', '--environment', action='store_true',
help='Set passwod, but from environment variable of '
'same name')
(options, args) = argparser.parse_args()
try:
noderange = args[0]
username = args[1]
except IndexError:
argparser.print_help()
sys.exit(1)
client.check_globbing(noderange)
session = client.Command()
exitcode = 0
if options.prompt:
oneval = 1
twoval = 2
while oneval != twoval:
oneval = getpass('Enter pass for {0}: '.format(username))
twoval = getpass('Confirm pass for {0}: '.format(username))
if oneval != twoval:
print('Values did not match.')
new_password = twoval
elif len(args) == 3:
if options.environment:
key = args[2]
new_password = os.environ.get(key, os.environ[key.upper()])
else:
new_password = args[2]
else:
argparser.print_help()
sys.exit(1)
errorNodes = set([])
uid_dict = {}
session.stop_if_noderange_over(noderange, options.maxnodes)
for rsp in session.read('/noderange/{0}/configuration/management_controller/users/all'.format(noderange)):
databynode = rsp["databynode"]
for node in databynode:
if 'error' in rsp['databynode'][node]:
print(node, ':', rsp['databynode'][node]['error'])
errorNodes.add(node)
continue
for user in rsp['databynode'][node]['users']:
if user['username'] == username:
if not user['uid'] in uid_dict:
uid_dict[user['uid']] = node
continue
uid_dict[user['uid']] = uid_dict[user['uid']] + ',{}'.format(node)
break
if not uid_dict:
print("Error: Could not reach target node's bmc user")
sys.exit(1)
for uid in uid_dict:
success = session.simple_noderange_command(uid_dict[uid], 'configuration/management_controller/users/{0}'.format(uid), new_password, key='password', errnodes=errorNodes) # = 0 if successful
allNodes = set([])
for node in session.read('/noderange/{0}/nodes/'.format(noderange)):
if 'error' in node and success != 0:
sys.exit(success)
allNodes.add(node['item']['href'].replace("/", ""))
goodNodes = allNodes - errorNodes
for node in goodNodes:
print(node + ": Password Change Successful")
sys.exit(success)
+2 -2
View File
@@ -47,14 +47,14 @@ exitcode = 0
errorNodes = set([])
session.stop_if_noderange_over(noderange, options.maxnodes)
success = session.simple_noderange_command(noderange, 'configuration/management_controller/reset', 'reset', key='state', errnodes=errorNodes) # = 0 if successful
if success != 0:
sys.exit(success)
# Determine which nodes were successful and print them
allNodes = set([])
for node in session.read('/noderange/{0}/nodes/'.format(noderange)):
if 'error' in node and success != 0:
sys.exit(success)
allNodes.add(node['item']['href'].replace("/", ""))
goodNodes = allNodes - errorNodes
+15 -29
View File
@@ -43,7 +43,7 @@ def bailout(msg, code=1):
sys.exit(code)
argparser = optparse.OptionParser(usage="Usage: %prog [options] <noderange> [setting|setting=value]")
argparser = optparse.OptionParser(usage="Usage: %prog [options] noderange [option|option=value]")
argparser.add_option('-c', '--comparedefault', dest='comparedefault',
action='store_true', default=False,
help='Compare given settings to default or list settings '
@@ -90,12 +90,6 @@ cfgpaths = {
'bmc.ipv4_gateway': (
'configuration/management_controller/net_interfaces/management',
'ipv4_gateway'),
'bmc.static_ipv6_addresses': (
'configuration/management_controller/net_interfaces/management',
'static_v6_addresses'),
'bmc.static_ipv6_gateway': (
'configuration/management_controller/net_interfaces/management',
'static_v6_gateway'),
'bmc.hostname': (
'configuration/management_controller/hostname', 'hostname'),
}
@@ -140,10 +134,9 @@ def _assign_value():
assignment[key] = value
def parse_config_line(arguments, single=False):
def parse_config_line(arguments):
global setmode, printallbmc, forceset, key, value, needval, candidate, path, attrib
for pidx in range(0, len(arguments)):
param = arguments[pidx]
for param in arguments:
if param == 'show':
continue # forgive muscle memory of pasu users
if param == 'set':
@@ -153,19 +146,14 @@ def parse_config_line(arguments, single=False):
if needval:
key = needval
needval = None
if single:
value = ' '.join(arguments[pidx:])
_assign_value()
break
else:
value = param
value = param
_assign_value()
continue
if '=' in param or param[-1] == ':' or forceset:
if setmode is None:
setmode = True
if setmode != True:
bailout('Cannot do set and query in same command: Query detected but "{0}" appears to be set'.format(param))
bailout('Cannot do set and query in same command')
if '=' in param:
key, _, value = param.partition('=')
_assign_value()
@@ -177,7 +165,7 @@ def parse_config_line(arguments, single=False):
if setmode is None:
setmode = False
if setmode != False:
bailout('Cannot do set and query in same command: Set mode detected but "{0}" appears to be a query'.format(param))
bailout('Cannot do set and query in same command')
if '.' not in param:
if param == 'bmc':
printallbmc = True
@@ -227,7 +215,7 @@ if options.batch:
pass
argset = argset.strip()
if argset:
parse_config_line(shlex.split(argset), single=True)
parse_config_line(shlex.split(argset))
argset = argfile.readline()
else:
parse_config_line(args[1:])
@@ -284,6 +272,7 @@ if setmode:
rcode |= client.printerror(fr)
for node in fr.get('databynode', []):
r = fr['databynode'][node]
rcode |= client.printerror(r, node)
if 'value' not in r:
continue
keyval = r['value']
@@ -295,22 +284,19 @@ else:
for path in queryparms:
if options.comparedefault:
continue
rcode |= client.print_attrib_path(path, session, list(queryparms[path]),
rc = client.print_attrib_path(path, session, list(queryparms[path]),
NullOpt(), queryparms[path])
if rc:
sys.exit(rc)
if printsys == 'all' or printextbmc or printbmc or printallbmc:
if printbmc or not printextbmc:
rcode |= client.print_attrib_path(
rcode = client.print_attrib_path(
'/noderange/{0}/configuration/management_controller/extended/all'.format(noderange),
session, printbmc, options, attrprefix='bmc.')
if options.extra:
if options.advanced:
rcode |= client.print_attrib_path(
'/noderange/{0}/configuration/management_controller/extended/extra_advanced'.format(noderange),
session, printextbmc, options)
else:
rcode |= client.print_attrib_path(
'/noderange/{0}/configuration/management_controller/extended/extra'.format(noderange),
session, printextbmc, options)
rcode |= client.print_attrib_path(
'/noderange/{0}/configuration/management_controller/extended/extra'.format(noderange),
session, printextbmc, options)
if printsys or options.exclude:
if printsys == 'all':
printsys = []
+3 -553
View File
@@ -1,4 +1,4 @@
#!/usr/libexec/platform-python
#!/usr/bin/python2
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2015 Lenovo
@@ -15,7 +15,6 @@
# See the License for the specific language governing permissions and
# limitations under the License.
import base64
import optparse
import os
import subprocess
@@ -24,565 +23,22 @@ path = os.path.dirname(os.path.realpath(__file__))
path = os.path.realpath(os.path.join(path, '..', 'lib', 'python'))
if path.startswith('/opt'):
sys.path.append(path)
import confluent.client as client
import confluent.sortutil as sortutil
import confluent.logreader as logreader
import time
import select
import socket
import re
import tty
import termios
import fcntl
import confluent.screensqueeze as sq
try:
from PIL import Image
except ImportError:
Image = None
try:
# sixel is optional, attempt to import but stub out if unavailable
import io
import sixel
class DumbWriter(sixel.SixelWriter):
def restore_position(self, output):
return
except ImportError:
class DumbWriter():
def draw(self, imgfile):
sys.stderr.write("PySixel not detected, Sixel format display not supported\n")
confettypath = os.path.join(os.path.dirname(sys.argv[0]), 'confetty')
argparser = optparse.OptionParser(
usage="Usage: %prog [options] <noderange> [kill][-- [passthroughoptions]]",
usage="Usage: %prog [options] node",
epilog="Command sequences are available while connected to a console, hit "
"ctrl-'e', then release ctrl, then 'c', then '?' for a full list. "
"For example, ctrl-'e', then 'c', then '.' will exit the current "
"console")
argparser.add_option('-t', '--tile', action='store_true', default=False,
help='Tile console windows in the terminal')
argparser.add_option('-l', '--log', action='store_true', default=False,
help='Enter log replay mode instead of showing a live console')
argparser.add_option('-T', '--Timestamp', action='store_true', default=False,
help= 'Dump log in stdout with timestamps')
argparser.add_option('-s', '--screenshot', action='store_true', default=False,
help='Attempt to grab screenshot and render using kitty image protocol')
argparser.add_option('-i', '--interval', type='float',
help='Interval in seconds to redraw the screenshot. Currently only '
'works for one node')
argparser.add_option('-w','--windowed', action='store_true', default=False,
help='Open terminal windows for each node. The '
'environment variable NODECONSOLE_WINDOWED_COMMAND '
'should be set, which should be a text string corresponding '
'to a command that can be used to open a windowed console,'
' omitting the "nodeconsole <noderange>" part of the '
'command, for example, to open a set of consoles for a '
'range of nodes in separate xterm windows, set '
'NODECONSOLE_WINDOWED_COMMAND to "xterm -e". To open a '
'set of consoles for a range of nodes in separate '
'GNOME Terminal windows with a size of 100 columns and '
'31 rows, set NODECONSOLE_WINDOWED_COMMAND '
'to "gnome-terminal --geometry 100x31 --" or in a WSL '
'environment, to open a set of consoles for a range of '
'nodes in separate Windows Terminal windows, with the '
'title set for each node, set NODECONSOLE_WINDOWED_COMMAND'
' to "wt.exe wsl.exe -d AlmaLinux-8 '
'--shell-type login". If the NODECONSOLE_WINDOWED_COMMAND '
'environment variable isn\'t set, xterm will be used by'
'default.')
(options, args) = argparser.parse_args()
oldtcattr = None
oldfl = None
def get_coords():
sys.stdout.write('\x1b[6n') #
sys.stdout.flush()
gotreply = select.select([sys.stdin,], [], [], 0.250)[0]
if gotreply:
response = ''
while select.select([sys.stdin,], [], [], 0.1)[0] and 'R' not in response:
response += sys.stdin.read()
coords = response.replace('R', '').split('[')[1].split(';')
#sys.stdout.write('\x1b[{}:{}H'.format(*coords))
def direct_console():
global oldtcattr
global oldfl
oldtcattr = termios.tcgetattr(sys.stdin.fileno())
oldfl = fcntl.fcntl(sys.stdin.fileno(), fcntl.F_GETFL)
tty.setraw(sys.stdin.fileno())
fcntl.fcntl(sys.stdin.fileno(), fcntl.F_SETFL, oldfl | os.O_NONBLOCK)
def indirect_console():
fcntl.fcntl(sys.stdin.fileno(), fcntl.F_SETFL, oldfl & ~os.O_NONBLOCK)
termios.tcsetattr(sys.stdin.fileno(), termios.TCSANOW, oldtcattr)
def determine_tile_size(numnodes):
# for now, smash everything to a common aspect ratio. 16:11
# is pretty much wrong for everything, making 4:3 a bit too wide
# and 16:9 significantly too narrow, but it is serviceable
# An improvement could come with us owning the scaling
# instead of delegating to Kitty, which says if we specify both,
# we get stretching. In theory we should be able to get aspect correct
# from kitty by omitting, but:
# then we don't know how much to move the cursor left after draw_image
# Konsole won't scale at all with only partial scaling specified
cheight, cwidth, pixwidth, pixheight = sq.get_screengeom()
# 16:12 is to roughly account for the 'titles' of the tiles
ratio = (pixwidth / 16) / (pixheight / 12)
bestdeviation = None
bestdims = []
for i in range(1, numnodes + 1):
number = numnodes
while number % i != 0:
number += 1
columns = i
rows = number // i
deviation = abs(ratio - (columns / rows))
if bestdeviation is None:
bestdeviation = deviation
bestdims = [columns, rows]
elif deviation < bestdeviation:
bestdeviation = deviation
bestdims = [columns, rows]
# ok, the above algorithm can still pick things like
# 1 2 3
# 4
# So we will let it pick the number of rows, and
# then see if we can chop columns and still fit
while (bestdims[0] - 1) * bestdims[1] >= numnodes:
bestdims[0] = bestdims[0] - 1
cellswide = cwidth // bestdims[0]
cellshigh = cheight // bestdims[1]
tilewidth = cellswide * pixwidth / cwidth
tileheight = cellshigh * pixheight / cheight
if tilewidth > (tileheight * 16 / 11):
tilewidth = tileheight * 16 / 11
cellswide = int(tilewidth // (pixwidth / cwidth))
if tileheight > (tilewidth * 11 /16):
tileheight = tilewidth * 11 / 16
cellshigh = int(tileheight // (pixheight / cheight))
bestdims = bestdims + [cellswide, cellshigh, cellshigh * bestdims[1]]
# incur any scrolling we might get. This allows us to accurately
# save/restore cursor or even get coordinates without scrolling fouling
# the desired target
sys.stdout.write('\n' * bestdims[4])
sys.stdout.flush()
cursor_up(bestdims[4])
return bestdims
cursor_saved = False
def sticky_cursor():
global cursor_saved
# get cursor restore_position
if sys.stdin.isatty() and not cursor_saved:
try:
direct_console()
sys.stdout.write('\x1b7')
cursor_saved = True
finally:
indirect_console()
elif cursor_saved:
try:
direct_console()
sys.stdout.write('\x1b8')
finally:
indirect_console()
def cursor_up(count=1):
sys.stdout.write(f'\x1b[{count}A')
def cursor_down(count=1):
sys.stdout.write(f'\x1b[{count}B')
def cursor_right(count=1):
sys.stdout.write(f'\x1b[{count}C')
def cursor_left(count=1):
sys.stdout.write(f'\x1b[{count}D')
def cursor_save():
sys.stdout.write('\x1b7')
def cursor_restore():
sys.stdout.write('\x1b8')
def cursor_hide():
sys.stdout.write('\x1b[?25l')
def cursor_show():
sys.stdout.write('\x1b[?25h')
def draw_image(data, width, height):
imageformat = os.environ.get('CONFLUENT_IMAGE_PROTOCOL', 'kitty')
if imageformat == 'sixel':
sixel_draw(data)
elif imageformat == 'iterm':
iterm_draw(data, width, height)
else:
kitty_draw(data, width, height)
def sixel_draw(data):
bindata = base64.b64decode(data)
binfile = io.BytesIO()
binfile.write(bindata)
binfile.seek(0)
DumbWriter().draw(binfile)
def iterm_draw(data, width, height):
if not height:
height = 'auto'
if not width:
width = 'auto'
bindata = base64.b64decode(data)
datalen = len(bindata)
sys.stdout.write(
'\x1b]1337;File=inline=1;width={};height={};size={}:'.format(width,height,datalen))
sys.stdout.write(data.decode('utf8'))
sys.stdout.write('\a')
sys.stdout.flush()
def kitty_draw(data, width, height):
if Image:
bindata = base64.b64decode(data)
binfile = io.BytesIO()
binfile.write(bindata)
binfile.seek(0)
img = Image.open(binfile)
outfile = io.BytesIO()
img.save(outfile, format='PNG')
data = base64.b64encode(outfile.getbuffer())
preamble = '\x1b_Ga=T,f=100'
if height:
preamble += f',r={height},c={width}'
#sys.stdout.write(repr(preamble))
#sys.stdout.write('\xb[{}D'.format(len(repr(preamble))))
#return
first = True
while data:
chunk, data = data[:4096], data[4096:]
m = 1 if data else 0
if first:
sys.stdout.write('{},m={};'.format(preamble, m))
else:
sys.stdout.write('\x1b_Gm={};'.format(m))
sys.stdout.write(chunk.decode('utf8'))
sys.stdout.write('\x1b\\')
sys.stdout.flush()
pass_through_args = []
killcon = False
try:
noderange = args[0]
if len(args) > 1:
if args[1] == 'kill':
killcon = True
pass_through_args = args[1:]
args = args[:1]
except IndexError:
argparser.print_help()
sys.exit(1)
if len(args) != 1:
argparser.print_help()
sys.exit(1)
if options.log:
logname = args[0]
if not os.path.exists(logname) and logname[0] != '/':
logname = os.path.join('/var/log/confluent/consoles', logname)
if not os.path.exists(logname):
sys.stderr.write('Unable to locate {0} on local system\n'.format(logname))
sys.exit(1)
logreader.replay_to_console(logname)
sys.exit(0)
if options.Timestamp:
logname = args[0]
if not os.path.exists(logname) and logname[0] != '/':
logname = os.path.join('/var/log/confluent/consoles', logname)
if not os.path.exists(logname):
sys.stderr.write('Unable to locate {0} on local system\n'.format(logname))
sys.exit(1)
logreader.dump_to_console(logname)
sys.exit(0)
def prep_node_tile(node):
currcolcell, currrowcell = nodepositions[node]
if currcolcell:
cursor_right(currcolcell)
if currrowcell:
cursor_down(currrowcell)
sys.stdout.write(node)
cursor_left(len(node))
cursor_down()
def reset_cursor(node):
currcolcell, currrowcell = nodepositions[node]
if currcolcell:
cursor_left(currcolcell)
cursor_up(currrowcell + 1)
nodepositions = {}
numrows = 0
def do_screenshot():
global numrows
cwidth = None
cheight = None
sess = client.Command()
if options.tile:
imageformat = os.environ.get('CONFLUENT_IMAGE_PROTOCOL', 'kitty')
if imageformat not in ('kitty', 'iterm'):
sys.stderr.write('Tiled screenshots only supported with kitty or iterm protocol')
sys.exit(1)
allnodes = []
numnodes = 0
for res in sess.read('/noderange/{}/nodes/'.format(args[0])):
allnodes.append(res['item']['href'].replace('/', ''))
numnodes += 1
cols, rows, cwidth, cheight, numrows = determine_tile_size(numnodes)
currcol = 1
currcolcell = 0
currrowcell = 0
for node in allnodes:
nodepositions[node] = currcolcell, currrowcell
if currcol < cols:
currcol += 1
currcolcell += cwidth
else:
currcol = 1
currcolcell = 0
currrowcell += cheight
elif options.interval is not None:
sys.stdout.write('\x1bc')
firstnodename = None
dorefresh = True
while dorefresh:
for res in sess.read('/noderange/{}/console/ikvm_screenshot'.format(args[0])):
for node in res.get('databynode', {}):
if not firstnodename:
firstnodename = node
imgdata = res['databynode'][node].get('image', {}).get('imgdata', None)
if imgdata:
if len(imgdata) < 32: # We were subjected to error
sys.stderr.write(f'{node}: Unable to get screenshot\n')
continue
if node in nodepositions:
prep_node_tile(node)
cursor_save()
else:
if options.interval is not None:
if node != firstnodename:
sys.stderr.write('Multiple nodes not supported for interval')
sys.exit(1)
sticky_cursor()
sys.stdout.write('{}: '.format(node))
# one row is used by our own name, so cheight - 1 for that allowance
draw_image(imgdata.encode(), cwidth, cheight - 1 if cheight else cheight)
if node in nodepositions:
cursor_restore()
reset_cursor(node)
else:
sys.stdout.write('\n')
sys.stdout.flush()
if options.interval is None:
dorefresh = False
else:
dorefresh = True
time.sleep(options.interval)
sys.exit(0)
if options.screenshot:
try:
cursor_hide()
do_screenshot()
except KeyboardInterrupt:
pass
finally:
cursor_show()
cursor_down(numrows)
sys.stdout.write('\n')
sys.exit(0)
def kill(noderange):
sess = client.Command()
envstring=os.environ.get('NODECONSOLE_WINDOWED_COMMAND')
if not envstring:
envstring = 'xterm'
nodes = []
for res in sess.read('/noderange/{0}/nodes/'.format(args[0])):
node = res.get('item', {}).get('href', '/').replace('/', '')
if not node:
sys.stderr.write(res.get('error', repr(res)) + '\n')
sys.exit(1)
nodes.append(node)
for node in nodes:
s=socket.socket(socket.AF_UNIX)
winid=None
try:
win=subprocess.Popen(['xwininfo', '-tree', '-root'], stdout=subprocess.PIPE)
wintr=win.communicate()[0]
for line in wintr.decode('utf-8').split('\n'):
if 'console: {0}'.format(node) in line or 'confetty' in line:
win_obj = [ele for ele in line.split(' ') if ele.strip()]
winid = win_obj[0]
except:
print("Error: cannot retrieve window id of node {}".format(node))
if winid:
ps_data=subprocess.Popen(['xkill', '-id', winid ], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
sys.exit(0)
def handle_geometry(envlist, sizegeometry, side_pad=0, top_pad=0, first=False):
if '-geometry' in envlist:
g_index = envlist.index('-geometry')
elif '-g' in envlist:
g_index = envlist.index('-g')
else:
g_index = 0
if g_index:
if first:
envlist[g_index+1] = '{0}+{1}+{2}'.format(envlist[g_index+1],side_pad, top_pad)
else:
envlist[g_index+1] = '{0}+{1}+{2}'.format(sizegeometry,side_pad, top_pad)
else:
envlist.insert(1, '-geometry')
envlist.insert(2, '{0}+{1}+{2}'.format(sizegeometry,side_pad, top_pad))
g_index = 1
return envlist
# add funcltionality to close/kill all open consoles
if killcon:
kill(noderange)
#added functionality for wcons
if options.windowed:
result=subprocess.Popen(['xwininfo', '-root'], stdout=subprocess.PIPE)
rootinfo=result.communicate()[0]
result.wait()
for line in rootinfo.decode('utf-8').split('\n'):
if 'Width' in line:
screenwidth = int(line.split(':')[1])
if 'Height' in line:
screenheight = int(line.split(':')[1])
envstring=os.environ.get('NODECONSOLE_WINDOWED_COMMAND')
if not envstring:
sizegeometry='100x31'
corrected_x, corrected_y = (13,84)
envlist = handle_geometry(['xterm'] + pass_through_args + ['-e'],sizegeometry, first=True)
#envlist=['xterm', '-bg', 'black', '-fg', 'white', '-geometry', '{sizegeometry}+0+0'.format(sizegeometry=sizegeometry), '-e']
else:
envlist=os.environ.get('NODECONSOLE_WINDOWED_COMMAND').split(' ')
if envlist[0] == 'xterm':
if '-geometry' in envlist:
g_index = envlist.index('-geometry')
elif '-g' in envlist:
g_index = envlist.index('-g')
else:
g_index = 0
if g_index:
envlist[g_index+1] = envlist[g_index+1] + '+0+0'
else:
envlist.insert(1, '-geometry')
envlist.insert(2, '100x31+0+0')
g_index = 1
nodes = []
sess = client.Command()
for res in sess.read('/noderange/{0}/nodes/'.format(args[0])):
node = res.get('item', {}).get('href', '/').replace('/', '')
if not node:
sys.stderr.write(res.get('error', repr(res)) + '\n')
sys.exit(1)
nodes.append(node)
if options.tile and not envlist[0] == 'xterm':
sys.stderr.write('[ERROR] UNSUPPORTED OPTIONS. \nWindowed and tiled consoles are only supported when using xterm \n')
sys.exit(1)
firstnode=nodes[0]
nodes.pop(0)
with open(os.devnull, 'wb') as devnull:
xopen=subprocess.Popen(envlist + [confettypath, '-c', '/tmp/controlpath-{0}'.format(firstnode), '-m', '5', 'start', '/nodes/{0}/console/session'.format(firstnode) ] , stdin=devnull)
time.sleep(2)
s=socket.socket(socket.AF_UNIX)
winid=''
try:
s.connect('/tmp/controlpath-{firstnode}'.format(firstnode=firstnode))
s.recv(64)
s.send(b'GETWINID')
winid=s.recv(64).decode('utf-8')
except:
time.sleep(2)
# try to get id of first panel/xterm window using name
win=subprocess.Popen(['xwininfo', '-tree', '-root'], stdout=subprocess.PIPE)
wintr=win.communicate()[0]
for line in wintr.decode('utf-8').split('\n'):
if 'console: {firstnode}'.format(firstnode=firstnode) in line or 'confetty' in line:
win_obj = [ele for ele in line.split(' ') if ele.strip()]
winid = win_obj[0]
if winid:
firstnode_window=subprocess.Popen(['xwininfo', '-id', '{winid}'.format(winid=winid)], stdout=subprocess.PIPE)
xinfo=firstnode_window.communicate()[0]
xinfl = xinfo.decode('utf-8').split('\n')
for line in xinfl:
if 'Absolute upper-left X:' in line:
side_pad = int(line.split(':')[1])
elif 'Absolute upper-left Y:' in line:
top_pad = int(line.split(':')[1])
elif 'Width:' in line:
window_width = int(line.split(':')[1])
elif 'Height' in line:
window_height = int(line.split(':')[1])
elif '-geometry' in line:
l = re.split(' |x|-|\\+', line)
l_nosp = [ele for ele in l if ele.strip()]
wmxo = int(l_nosp[1])
wmyo = int(l_nosp[2])
sizegeometry = str(wmxo) + 'x' + str(wmyo)
else:
pass
window_width += side_pad*2
window_height += side_pad+top_pad
screenwidth -= wmxo
screenheight -= wmyo
currx = window_width
curry = 0
maxcol = int(screenwidth/window_width)
for node in sortutil.natural_sort(nodes):
if options.tile and envlist[0] == 'xterm':
corrected_x = currx
corrected_y = curry
xgeometry = '{0}+{1}+{2}'.format(sizegeometry, corrected_x, corrected_y)
currx += window_width
if currx >= screenwidth:
currx=0
curry += window_height
if curry > screenheight:
curry =top_pad
if not envstring:
envlist= handle_geometry(envlist, sizegeometry, corrected_x, corrected_y)
else:
if g_index:
envlist[g_index+1] = xgeometry
elif envlist[0] == 'xterm':
envlist=handle_geometry(envlist, sizegeometry, side_pad, top_pad)
side_pad+=(side_pad+1)
top_pad+=(top_pad+30)
else:
pass
with open(os.devnull, 'wb') as devnull:
xopen=subprocess.Popen(envlist + [confettypath, '-c', '/tmp/controlpath-{0}'.format(node), '-m', '5', 'start', '/nodes/{0}/console/session'.format(node)] , stdin=devnull)
sys.exit(0)
#end of wcons
if options.tile:
null = open('/dev/null', 'w')
nodes = []
@@ -594,13 +50,8 @@ if options.tile:
sys.exit(1)
nodes.append(node)
initial = True
in_tmux = False
pane = 0
sessname = 'nodeconsole_{0}'.format(os.getpid())
if os.environ.get("TMUX"):
initial = False
in_tmux = True
subprocess.call(['tmux', 'rename-session', sessname])
for node in sortutil.natural_sort(nodes):
panename = '{0}:{1}'.format(sessname, pane)
if initial:
@@ -621,8 +72,7 @@ if options.tile:
pane += 1
subprocess.call(['tmux', 'select-pane', '-t', sessname])
subprocess.call(['tmux', 'set-option', '-t', panename, 'pane-border-status', 'top'], stderr=null)
if not in_tmux:
os.execlp('tmux', 'tmux', 'attach', '-t', sessname)
os.execlp('tmux', 'tmux', 'attach', '-t', sessname)
else:
os.execl(confettypath, confettypath, 'start',
'/nodes/{0}/console/session'.format(args[0]))
-178
View File
@@ -1,178 +0,0 @@
#!/usr/bin/python3
import argparse
import os
import sys
path = os.path.dirname(os.path.realpath(__file__))
path = os.path.realpath(os.path.join(path, '..', 'lib', 'python'))
if path.startswith('/opt'):
sys.path.append(path)
import confluent.client as client
import confluent.sortutil as sortutil
def cleararm(nr, cli):
nodes = set([])
for rsp in cli.read('/noderange/{0}/attributes/current'.format(nr)):
for node in rsp.get('databynode', {}):
nodeinfo = rsp['databynode'][node]
for attr in nodeinfo:
if attr == 'deployment.apiarmed':
curr = nodeinfo[attr].get('value', '')
if curr == 'continuous':
nodes.add(node)
noderange = nr
if nodes:
noderange += ',-({0})'.format(','.join(nodes))
for rsp in cli.update('/noderange/{0}/attributes/current'.format(noderange),
{'deployment.apiarmed': ''}):
pass
def armonce(nr, cli):
nodes = set([])
for rsp in cli.read('/noderange/{0}/attributes/current'.format(nr)):
for node in rsp.get('databynode', {}):
nodeinfo = rsp['databynode'][node]
for attr in nodeinfo:
if attr == 'deployment.apiarmed':
curr = nodeinfo[attr].get('value', '')
if curr == 'continuous':
nodes.add(node)
noderange = nr
if nodes:
noderange += ',-({0})'.format(','.join(nodes))
for rsp in cli.update('/noderange/{0}/attributes/current'.format(noderange),
{'deployment.apiarmed': 'once'}):
pass
def setpending(nr, profile, cli):
args = {'deployment.pendingprofile': profile, 'deployment.state': '', 'deployment.state_detail': ''}
if not profile.startswith('genesis-'):
args['deployment.stagedprofile'] = ''
args['deployment.profile'] = ''
for rsp in cli.update('/noderange/{0}/attributes/current'.format(nr),
args):
pass
def clearpending(nr, cli):
for rsp in cli.update('/noderange/{0}/attributes/current'.format(nr),
{'deployment.pendingprofile': ''}):
pass
def main(args):
ap = argparse.ArgumentParser(description='Deploy OS to nodes')
ap.add_argument('-c', '--clear', help='Clear any pending deployment action', action='store_true')
ap.add_argument('-n', '--network', help='Initiate deployment over PXE/HTTP', action='store_true')
ap.add_argument('-p', '--prepareonly', help='Prepare only, skip any interaction with a BMC associated with this deployment action', action='store_true')
ap.add_argument('-m', '--maxnodes', help='Specifiy a maximum nodes to be deployed')
ap.add_argument('noderange', help='Set of nodes to deploy')
ap.add_argument('profile', nargs='?', help='Profile name to deploy')
args, extra = ap.parse_known_args(args)
if args.profile is None and len(extra) == 1:
args.profile = extra[0]
extra = extra[1:]
if args.profile and not args.network:
sys.stderr.write('-n is a required argument currently to perform an install, optionally with -p\n')
return 1
if not args.profile and args.network:
sys.stderr.write('Both noderange and a profile name are required arguments to request a network deployment\n')
return 1
if args.clear and args.profile:
sys.stderr.write(
'The -c/--clear option should not be used with a profile, '
'it is a request to not deploy any profile, and will clear '
'whatever the current profile is without being specified\n')
return 1
if extra:
sys.stderr.write('Unrecognized arguments: ' + repr(extra) + '\n')
c = client.Command()
c.stop_if_noderange_over(args.noderange, args.maxnodes)
errnodes = set([])
for rsp in c.read('/noderange/{0}/nodes/'.format(args.noderange)):
if 'error' in rsp:
sys.stderr.write(rsp['error'] + '\n')
sys.exit(1)
if args.clear:
cleararm(args.noderange, c)
clearpending(args.noderange, c)
elif args.profile:
profnames = []
for prof in c.read('/deployment/profiles/'):
profname = prof.get('item', {}).get('href', None)
if profname:
profname = profname.replace('/', '')
profnames.append(profname)
if profname == args.profile:
break
else:
sys.stderr.write('The specified profile "{}" is not an available profile\n'.format(args.profile))
if profnames:
sys.stderr.write('The following profiles are available:\n')
for profname in profnames:
sys.stderr.write(' ' + profname + '\n')
else:
sys.stderr.write('No deployment profiles available, try osdeploy import or imgutil capture\n')
sys.exit(1)
armonce(args.noderange, c)
setpending(args.noderange, args.profile, c)
else:
databynode = {}
for r in c.read('/noderange/{0}/attributes/current'.format(args.noderange)):
dbn = r.get('databynode', {})
for node in dbn:
if node not in databynode:
databynode[node] = {}
for attr in dbn[node]:
if attr in ('deployment.pendingprofile', 'deployment.apiarmed', 'deployment.stagedprofile', 'deployment.profile', 'deployment.state', 'deployment.state_detail'):
databynode[node][attr] = dbn[node][attr].get('value', '')
for node in sortutil.natural_sort(databynode):
profile = databynode[node].get('deployment.pendingprofile', '')
if profile:
profile = 'pending: {}'.format(profile)
else:
profile = databynode[node].get('deployment.stagedprofile', '')
if profile:
profile = 'staged: {}'.format(profile)
else:
profile = databynode[node].get('deployment.profile', '')
if profile:
profile = 'completed: {}'.format(profile)
else:
profile= 'No profile pending or applied'
armed = databynode[node].get('deployment.apiarmed', '')
if armed in ('once', 'continuous'):
armed = ' (node authentication armed)'
else:
armed = ''
stateinfo = ''
deploymentstate = databynode[node].get('deployment.state', '')
if deploymentstate:
statedetails = databynode[node].get('deployment.state_detail', '')
if statedetails:
stateinfo = '{}: {}'.format(deploymentstate, statedetails)
else:
stateinfo = deploymentstate
if stateinfo:
print('{0}: {1} ({2})'.format(node, profile, stateinfo))
else:
print('{0}: {1}{2}'.format(node, profile, armed))
sys.exit(0)
if not args.clear and args.network and not args.prepareonly:
rc = c.simple_noderange_command(args.noderange, '/boot/nextdevice', 'network',
bootmode='uefi',
persistent=False,
errnodes=errnodes)
if errnodes:
sys.stderr.write(
'Unable to set boot device for following nodes: {0}\n'.format(
','.join(errnodes)))
return 1
rc |= c.simple_noderange_command(args.noderange, '/power/state', 'boot')
return 0
if __name__ == '__main__':
sys.exit(main(sys.argv[1:]))
+9 -71
View File
@@ -34,7 +34,6 @@ defcolumns = ['Node', 'Model', 'Serial', 'UUID', 'Mac Address', 'Type',
columnmapping = {
'Node': 'nodename',
'Model': 'modelnumber',
'Model Name': 'modelname',
'Serial': 'serialnumber',
'UUID': 'uuid',
'Type': 'types',
@@ -48,54 +47,20 @@ columnmapping = {
'Advertised IP': 'otheripaddrs',
'Other IP': 'otheripaddrs',
}
#TODO: add chassis uuid
def register_endpoint(options, session, addr):
neednewline = False
current = 0
for rsp in session.update('/discovery/register', {'addresses': addr}):
if 'count' in rsp:
total = rsp['count']
elif total > 1:
current += 1
sys.stdout.write('\x1b[2K\r{0:.1f}% Scanned'.format(100 * current / total))
sys.stdout.flush()
neednewline = True
if 'created' in rsp:
if neednewline:
print('')
neednewline = False
print('Registered: {0}'.format(rsp['created']))
if neednewline:
print('')
def subscribe_discovery(options, session, subscribe, targ):
keyn = 'subscribe' if subscribe else 'unsubscribe'
payload = {keyn: targ}
if subscribe:
for rsp in session.update('/discovery/subscriptions/{0}'.format(targ), payload):
if 'status' in rsp:
print(rsp['status'])
else:
for rsp in session.delete('/discovery/subscriptions/{0}'.format(targ)):
if 'status' in rsp:
print(rsp['status'])
def print_disco(options, session, currmac, outhandler, columns):
procinfo = {}
for tmpinfo in session.read('/discovery/by-mac/{0}'.format(currmac)):
procinfo.update(tmpinfo)
if 'Switch' in columns or 'Port' in columns:
if 'switch' in procinfo:
procinfo['port'] = procinfo['switchport']
else:
for tmpinfo in session.read(
'/networking/macs/by-mac/{0}'.format(currmac)):
if 'ports' in tmpinfo:
# The api sorts so that the most specific available value
# is last
procinfo.update(tmpinfo['ports'][-1])
for tmpinfo in session.read(
'/networking/macs/by-mac/{0}'.format(currmac)):
if 'ports' in tmpinfo:
# The api sorts so that the most specific available value
# is last
procinfo.update(tmpinfo['ports'][-1])
record = []
for col in columns:
rawcol = columnmapping[col]
@@ -183,7 +148,6 @@ unique_fields = frozenset(['serial', 'mac', 'uuid'])
def import_csv(options, session):
nodedata = []
unique_data = {}
exitcode = 0
with open(options.importfile, 'r') as datasrc:
records = csv.reader(datasrc)
fields = process_header(next(records))
@@ -224,7 +188,6 @@ def import_csv(options, session):
for res in session.create('/nodes/', datum):
if 'error' in res:
sys.stderr.write(res['error'] + '\n')
exitcode |= res.get('errorcode', 1)
continue
elif 'created' in res:
print('Defined ' + res['created'])
@@ -235,14 +198,11 @@ def import_csv(options, session):
{'node': nodename}):
if 'error' in res:
sys.stderr.write(res['error'] + '\n')
exitcode |= res.get('errorcode', 1)
continue
elif 'assigned' in res:
print('Discovered ' + res['assigned'])
else:
print(repr(res))
if exitcode:
sys.exit(exitcode)
def list_discovery(options, session):
@@ -303,11 +263,7 @@ def list_matching_macs(options, session, node=None, checknode=True):
return [options.mac.replace(':', '-')]
else:
path += 'by-mac/'
ret = []
for x in session.read(path):
if 'item' in x and 'href' in x['item']:
ret.append(x['item']['href'])
return ret
return [x['item']['href'] for x in session.read(path)]
def assign_discovery(options, session, needid=True):
abort = False
@@ -331,18 +287,12 @@ def assign_discovery(options, session, needid=True):
if not matches:
sys.stderr.write("No matching discovery candidates found\n")
sys.exit(1)
exitcode = 0
for res in session.update('/discovery/by-mac/{0}'.format(matches[0]),
{'node': options.node}):
if 'assigned' in res:
print('Assigned: {0}'.format(res['assigned']))
elif 'error' in res:
sys.stderr.write('Error: {0}\n'.format(res['error']))
exitcode |= res.get('errorcode', 1)
else:
print(repr(res))
if exitcode:
sys.exit(exitcode)
def blocking_scan(session):
list(session.update('/discovery/rescan', {'rescan': 'start'}))
@@ -353,7 +303,7 @@ def blocking_scan(session):
def main():
parser = optparse.OptionParser(
usage='Usage: %prog [list|assign|rescan|clear|subscribe|unsubscribe|register] [options]')
usage='Usage: %prog [list|assign|rescan|clear] [options]')
# -a for 'address' maybe?
# order by
# show state (discovered or..
@@ -391,15 +341,9 @@ def main():
parser.add_option('-o', '--order', dest='order',
help='Order output by given field', metavar='ORDER')
(options, args) = parser.parse_args()
if len(args) == 0 or args[0] not in ('list', 'assign', 'reassign', 'rescan', 'clear', 'subscribe', 'unsubscribe', 'register'):
if len(args) == 0 or args[0] not in ('list', 'assign', 'reassign', 'rescan', 'clear'):
parser.print_help()
sys.exit(1)
if args[0] == 'register' and len(args) != 2:
sys.stderr.write('Register requires target address or range (e.g. 192.168.2.30, 192.168.2.0/24, or 192.168.2.1-192.168.2.30)\n')
sys.exit(1)
if 'subscribe' in args[0] and len(args) != 2:
sys.stderr.write('subscribe/unsubscribe subcommands require switch name as argument\n')
sys.exit(1)
session = client.Command()
if args[0] == 'list':
list_discovery(options, session)
@@ -409,12 +353,6 @@ def main():
assign_discovery(options, session)
if args[0] == 'reassign':
assign_discovery(options, session, False)
if args[0] == 'register':
register_endpoint(options, session, args[1])
if args[0] == 'subscribe':
subscribe_discovery(options, session, True, args[1])
if args[0] == 'unsubscribe':
subscribe_discovery(options, session, False, args[1])
if args[0] == 'rescan':
blocking_scan(session)
print("Rescan complete")
+8 -70
View File
@@ -17,7 +17,6 @@
import codecs
from datetime import datetime as dt
from datetime import timedelta
import optparse
import os
import signal
@@ -38,22 +37,11 @@ if sys.version_info[0] < 3:
sys.stdout = codecs.getwriter('utf8')(sys.stdout)
argparser = optparse.OptionParser(
usage="Usage: %prog [options] <noderange> [clear]")
usage="Usage: %prog [options] noderange [clear]")
argparser.add_option('-m', '--maxnodes', type='int',
help='Specify a maximum number of '
'nodes to clear if clearing log, '
'prompting if over the threshold')
argparser.add_option('-l', '--lines', type='int',
help='return the last <n> entries '
'for each node in the eventlog. '
)
argparser.add_option('-t', '--timeframe', type='string',
help='return entries within a specified timeframe '
'for each node in the eventlog. This will return '
'entries from the last hours or days. '
'1h would be one hour, 4d would be four days. '
'format <num>h or <num>d'
)
'prompting if over the threshold')
(options, args) = argparser.parse_args()
try:
noderange = args[0]
@@ -78,11 +66,8 @@ exitcode = 0
def format_event(evt):
retparts = []
if 'timestamp' in evt and evt['timestamp'] is not None:
try:
display = dt.strptime(evt['timestamp'], '%Y-%m-%dT%H:%M:%S')
retparts.append(display.strftime('%m/%d/%Y %H:%M:%S'))
except ValueError:
display = ''
display = dt.strptime(evt['timestamp'], '%Y-%m-%dT%H:%M:%S')
retparts.append(display.strftime('%m/%d/%Y %H:%M:%S'))
dscparts = []
if evt.get('log_id', None):
retparts.append(evt['log_id'] + ':')
@@ -104,34 +89,12 @@ def format_event(evt):
msg = ''
return ' '.join(retparts) + msg
if deletemode:
func = session.delete
session.stop_if_noderange_over(noderange, options.maxnodes)
else:
func = session.read
if options.timeframe:
try:
delta = int(options.timeframe[:-1])
except ValueError:
argparser.print_help()
sys.exit(1)
if options.timeframe[-1].lower() == 'd':
tdelta = timedelta(days=delta)
elif options.timeframe[-1].lower() == 'h':
tdelta = timedelta(hours=delta)
else:
argparser.print_help()
sys.exit(1)
timeframe = dt.now() - tdelta
event_dict = {}
nodes = []
for res in session.read('/noderange/{0}/nodes/'.format(args[0])):
node = res.get('item', {}).get('href', '/').replace('/', '')
nodes.append(node)
event_dict[node] = []
for rsp in func('/noderange/{0}/events/hardware/log'.format(noderange)):
if 'error' in rsp:
sys.stderr.write(rsp['error'] + '\n')
@@ -144,31 +107,6 @@ for rsp in func('/noderange/{0}/events/hardware/log'.format(noderange)):
sys.stderr.write('{0}: {1}\n'.format(node, thisdata['error']))
exitcode |= 1
if 'events' in thisdata:
evtdata = thisdata['events']
if options.lines:
event_dict[node].extend(evtdata)
else:
for evt in evtdata:
if options.timeframe:
# check if line is in timeframe
if 'timestamp' in evt and evt['timestamp'] is not None:
display = dt.strptime(evt['timestamp'], '%Y-%m-%dT%H:%M:%S')
if display > timeframe:
print('{0}: {1}'.format(node, format_event(evt)))
else:
print('{0}: {1}'.format(node, format_event(evt)))
if options.lines:
for node in nodes:
evtdata_list = event_dict[node]
if len(evtdata_list) != 0:
if len(evtdata_list) > options.lines:
evtdata_list = evtdata_list[-abs(options.lines):]
for evt in evtdata_list:
if options.timeframe:
if 'timestamp' in evt and evt['timestamp'] is not None:
display = dt.strptime(evt['timestamp'], '%Y-%m-%dT%H:%M:%S')
if display > timeframe:
print('{0}: {1}'.format(node, format_event(evt)))
else:
print('{0}: {1}'.format(node, format_event(evt)))
evtdata = thisdata['events']
for evt in evtdata:
print('{0}: {1}'.format(node, format_event(evt)))
+7 -29
View File
@@ -56,7 +56,7 @@ components = ['all']
argparser = optparse.OptionParser(
usage="Usage: "
"%prog <noderange> [list][updatestatus][update [--backup <file>]]|[<components>]")
"%prog <noderange> [list][update [--backup <file>]]|[<components>]")
argparser.add_option('-b', '--backup', action='store_true',
help='Target a backup bank rather than primary')
argparser.add_option('-m', '--maxnodes', type='int',
@@ -65,18 +65,14 @@ argparser.add_option('-m', '--maxnodes', type='int',
(options, args) = argparser.parse_args()
upfile = None
querystatus = False
try:
noderange = args[0]
if len(args) > 1:
if args[1] == 'update':
upfile = args[2]
else:
comps = []
if args[1] == 'list':
comps = args[2:]
elif args[1] == 'updatestatus':
querystatus = True
else:
comps = args[1:]
components = []
@@ -92,9 +88,9 @@ client.check_globbing(noderange)
def get_update_progress(session, url):
for res in session.read(url):
status = res.get('phase', 'error')
percent = res.get('progress', None)
detail = res.get('detail', repr(res)),
status = res['phase']
percent = res['progress']
detail = res['detail']
if status == 'error':
text = 'error!'
else:
@@ -113,12 +109,6 @@ def update_firmware(session, filename):
if options.backup:
upargs['bank'] = 'backup'
noderrs = {}
if session.unixdomain:
of = open(filename, 'rb')
try:
session.add_file(filename, of.fileno(), 'rb')
except Exception:
pass
for res in session.create(resource, upargs):
if 'created' not in res:
for nodename in res.get('databynode', ()):
@@ -151,40 +141,28 @@ def update_firmware(session, filename):
def show_firmware(session):
global exitcode
firmware_shown = False
nodes_matched = False
for component in components:
for res in session.read(
'/noderange/{0}/inventory/firmware/all/{1}'.format(
noderange, component)):
nodes_matched = True
exitcode |= client.printerror(res)
if 'databynode' not in res:
continue
for node in res['databynode']:
exitcode |= client.printerror(res['databynode'][node], node)
if 'firmware' not in res['databynode'][node]:
continue
for inv in res['databynode'][node]['firmware']:
for prefix in inv:
firmware_shown = True
printfirm(node, prefix, inv[prefix])
if not nodes_matched:
sys.stderr.write('No matching nodes for noderange "{0}"\n'.format(noderange))
elif not firmware_shown and not exitcode:
if not firmware_shown and not exitcode:
argparser.print_help()
try:
session = client.Command()
if querystatus:
for res in session.read(
'/noderange/{0}/inventory/firmware/updatestatus'.format(noderange)):
for node in res.get('databynode', {}):
currstat = res['databynode'][node].get('status', None)
if currstat:
print('{}: {}'.format(node, currstat))
else:
print(repr(res))
elif upfile is None:
if upfile is None:
show_firmware(session)
else:
update_firmware(session, upfile)
+1 -3
View File
@@ -60,12 +60,10 @@ nodetype="nodegroups"
try:
nodegroups = args[0]
client.check_globbing(nodegroups)
nodelist = '/{0}/{1}/'.format(nodetype,nodegroups)
except IndexError:
argparser.print_help()
sys.exit(1)
nodelist = '/nodegroups/'
client.check_globbing(nodegroups)
session = client.Command()
exitcode = 0
+10 -11
View File
@@ -45,15 +45,14 @@ except IndexError:
client.check_globbing(noderange)
session = client.Command()
exitcode = 0
for group in noderange.split(','):
attribs = {'name': group}
for arg in args[1:]:
key, val = arg.split('=', 1)
attribs[key] = val
for r in session.create('/nodegroups/', attribs):
if 'error' in r:
sys.stderr.write(r['error'] + '\n')
exitcode |= 1
if 'created' in r:
print('{0}: created'.format(r['created']))
attribs = {'name': noderange}
for arg in args[1:]:
key, val = arg.split('=', 1)
attribs[key] = val
for r in session.create('/nodegroups/', attribs):
if 'error' in r:
sys.stderr.write(r['error'] + '\n')
exitcode |= 1
if 'created' in r:
print('{0}: created'.format(r['created']))
sys.exit(exitcode)
+6 -7
View File
@@ -43,11 +43,10 @@ noderange = args[0]
client.check_globbing(noderange)
session = client.Command()
exitcode = 0
for group in noderange.split(','):
for r in session.delete('/nodegroups/{0}'.format(group)):
if 'error' in r:
sys.stderr.write(r['error'] + '\n')
exitcode |= 1
if 'deleted' in r:
print('{0}: deleted'.format(r['deleted']))
for r in session.delete('/nodegroups/{0}'.format(noderange)):
if 'error' in r:
sys.stderr.write(r['error'] + '\n')
exitcode |= 1
if 'deleted' in r:
print('{0}: deleted'.format(r['deleted']))
sys.exit(exitcode)
-1
View File
@@ -50,7 +50,6 @@ codemappings = {
'warning': 1,
'critical': 2,
'failed': 2,
'unknown': 2,
}
def main():
+4 -35
View File
@@ -53,11 +53,6 @@ def print_mem_info(node, prefix, meminfo):
memdescfmt += '3-{1} '
elif 'DDR4' in meminfo['memory_type']:
memdescfmt += '4-{1} '
elif 'DDR5' in meminfo['memory_type']:
memdescfmt += '5-{1} '
elif 'DCPMM' in meminfo['memory_type']:
memdescfmt = '{0}GB {1} '
meminfo['module_type'] = 'DCPMM'
else:
print('{0}: {1}: Unrecognized Memory'.format(node, prefix))
return
@@ -101,7 +96,6 @@ usedprefixes = set([])
argparser = optparse.OptionParser(
usage="Usage: %prog <noderange> [serial|model|uuid|mac]")
argparser.add_option('-j', '--json', action='store_true', help='Output JSON')
argparser.add_option('-s', '--store', action='store_true', help='Store serial, model, and uuid into id.serial, id.model, and id.uuid')
(options, args) = argparser.parse_args()
try:
noderange = args[0]
@@ -129,36 +123,17 @@ if len(args) > 1:
try:
if options.json:
databynode = {}
if options.store and len(args) <= 1:
url = '/noderange/{0}/inventory/hardware/all/system'
pushattribs = {}
session = client.Command()
for res in session.read(url.format(noderange)):
printerror(res)
if 'databynode' not in res:
continue
for node in res['databynode']:
if options.store and node not in pushattribs:
pushattribs[node] = {}
printerror(res['databynode'][node], node)
if 'inventory' not in res['databynode'][node]:
continue
for inv in res['databynode'][node]['inventory']:
prefix = inv['name']
if options.store and prefix == 'System':
currinfo = inv.get('information', {})
curruuid = currinfo.get('UUID', '')
if curruuid:
curruuid = curruuid.lower()
pushattribs[node]['id.uuid'] = curruuid
currserial = currinfo.get('Serial Number', '')
if currserial:
currserial = currserial.strip()
pushattribs[node]['id.serial'] = currserial
currmodelnum = currinfo.get('Model', '')
if currmodelnum:
currmodelnum = currmodelnum.strip()
pushattribs[node]['id.model'] = currmodelnum
idx = 2
while (node, prefix) in usedprefixes:
prefix = '{0} {1}'.format(inv['name'], idx)
@@ -170,7 +145,7 @@ try:
if node not in databynode:
databynode[node] = {}
databynode[node][prefix] = inv
elif not options.store:
else:
print('{0}: {1}: Not Present'.format(node, prefix))
continue
info = inv['information']
@@ -201,18 +176,12 @@ try:
databynode[node] = {}
databynode[node][prefix] = inv
break
elif not options.store:
print(u'{0}: {1} {2}: {3}'.format(node, prefix,
pretty(datum),
info[datum]))
print(u'{0}: {1} {2}: {3}'.format(node, prefix,
pretty(datum),
info[datum]))
if options.json:
print(json.dumps(databynode, sort_keys=True, indent=4,
separators=(',', ': ')))
if pushattribs:
for node in pushattribs:
for rsp in session.update('/nodes/{0}/attributes/current'.format(node), pushattribs[node]):
if 'error' in rsp:
sys.stderr.write(rsp['error'] + '\n')
except KeyboardInterrupt:
print('')
sys.exit(exitcode)
-176
View File
@@ -1,176 +0,0 @@
#!/usr/libexec/platform-python
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2017 Lenovo
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
__author__ = 'tkucherera'
import optparse
import os
import signal
import sys
import subprocess
try:
signal.signal(signal.SIGPIPE, signal.SIG_DFL)
except AttributeError:
pass
path = os.path.dirname(os.path.realpath(__file__))
path = os.path.realpath(os.path.join(path, '..', 'lib', 'python'))
if path.startswith('/opt'):
sys.path.append(path)
import confluent.client as client
argparser = optparse.OptionParser(
usage="Usage: %prog <start_node> -i <interface> <end_node> -e <eface>",
)
argparser.add_option('-i', '--interface', type='str',
help='interface to check path against for the start node')
argparser.add_option('-e', '--eface', type='str',
help='interface to check path against for the end node')
argparser.add_option('-c', '--cumulus', action="store_true", dest="cumulus",
help='return layer 2 route through cumulus switches only')
(options, args) = argparser.parse_args()
try:
start_node = args[0]
end_node = args[1]
interface = options.interface
eface = options.eface
except IndexError:
argparser.print_help()
sys.exit(1)
session = client.Command()
def get_neighbors(switch):
switch_neigbors = []
url = '/networking/neighbors/by-switch/{0}/by-peername/'.format(switch)
for neighbor in session.read(url):
try:
switch = neighbor['item']['href'].strip('/')
except:
continue
if switch in all_switches:
switch_neigbors.append(switch)
return switch_neigbors
def find_path(start, end, path=[]):
path = path + [start]
if start == end:
return path # If start and end are the same, return the path
for node in get_neighbors(start):
if node not in path:
new_path = find_path(node, end, path)
if new_path:
return new_path # If a path is found, return it
return None # If no path is found, return None
def is_cumulus(switch):
try:
read_attrib = subprocess.check_output(['nodeattrib', switch, 'hardwaremanagement.method'])
except subprocess.CalledProcessError:
return False
for attribs in read_attrib.decode('utf-8').split('\n'):
if len(attribs.split(':')) > 1:
attrib = attribs.split(':')
if attrib[2].strip() == 'affluent':
return True
else:
return False
else:
return False
def host_to_switch(node, interface=None):
# first check the the node config to see what switches are connected
# if host is in rhel can use nmstate package
if node in all_switches:
return [node]
switches = []
netarg = 'net.*.switch'
if interface:
netarg = 'net.{0}.switch'.format(interface)
try:
read_attrib = subprocess.check_output(['nodeattrib', node, netarg])
except subprocess.CalledProcessError:
return False
for attribs in read_attrib.decode('utf-8').split('\n'):
attrib = attribs.split(':')
try:
if ' net.mgt.switch' in attrib or attrib[2] == '':
continue
except IndexError:
continue
switch = attrib[2].strip()
if is_cumulus(switch) and options.cumulus:
switches.append(switch)
else:
switches.append(switch)
return switches
def path_between_nodes(start_switches, end_switches):
for start_switch in start_switches:
for end_switch in end_switches:
if start_switch == end_switch:
return [start_switch]
else:
path = find_path(start_switch, end_switch)
if path:
return path
else:
return 'No path found'
all_switches = []
for res in session.read('/networking/neighbors/by-switch/'):
if 'error' in res:
sys.stderr.write(res['error'] + '\n')
exitcode = 1
else:
switch = (res['item']['href'].replace('/', ''))
all_switches.append(switch)
end_nodeslist = []
nodelist = '/noderange/{0}/nodes/'.format(end_node)
for res in session.read(nodelist):
if 'error' in res:
sys.stderr.write(res['error'] + '\n')
exitcode = 1
else:
elem=(res['item']['href'].replace('/', ''))
end_nodeslist.append(elem)
start_switches = host_to_switch(start_node, interface)
for end_node in end_nodeslist:
if end_node:
end_switches = host_to_switch(end_node, eface)
if not end_switches:
print('Error: net.{0}.switch attribute is not valid')
continue
path = path_between_nodes(start_switches, end_switches)
print(f'{start_node} to {end_node}: {path}')
# TODO dont put switches that are connected through management interfaces.
+1 -7
View File
@@ -36,7 +36,7 @@ exitcode = 0
argparser = optparse.OptionParser(
usage="Usage: "
"%prog <noderange> [list][install <license_filename>|save <directory>|delete <license_feature_name>]")
"%prog <noderange> [list][install <file>|save <directory>|delete <name>]")
argparser.add_option('-m', '--maxnodes', type='int',
help='Specify a maximum number of '
'nodes to delete licenses from, '
@@ -69,12 +69,6 @@ def install_license(session, filename):
'management_controller/licenses/'.format(noderange)
filename = os.path.abspath(filename)
instargs = {'filename': filename}
if session.unixdomain:
of = open(filename, 'rb')
try:
session.add_file(filename, of.fileno(), 'rb')
except Exception:
pass
for res in session.create(resource, instargs):
for node in res.get('databynode', []):
if 'error' in res['databynode'][node]:
+3 -10
View File
@@ -1,4 +1,4 @@
#!/usr/libexec/platform-python
#!/usr/bin/python2
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2015-2017 Lenovo
@@ -15,15 +15,13 @@
# See the License for the specific language governing permissions and
# limitations under the License.
__author__ = 'jjohnson2,alin37,wgrzeda'
__author__ = 'jjohnson2,alin37'
import optparse
import os
import signal
import sys
try:
signal.signal(signal.SIGPIPE, signal.SIG_DFL)
except AttributeError:
@@ -41,12 +39,9 @@ def main():
" or: %prog [options] noderange <nodeattribute>...")
argparser.add_option('-b', '--blame', action='store_true',
help='Show information about how attributes inherited')
argparser.add_option('-d', '--delim', metavar="STRING", default = "\n",
help='Delimiter separating the values')
(options, args) = argparser.parse_args()
noderange=""
nodelist=""
list = []
try:
noderange = args[0]
nodelist = '/noderange/{0}/nodes/'.format(noderange)
@@ -66,9 +61,7 @@ def main():
sys.stderr.write(res['error'] + '\n')
exitcode = 1
else:
elem=(res['item']['href'].replace('/', ''))
list.append(elem)
print(options.delim.join(list))
print(res['item']['href'].replace('/', ''))
sys.exit(exitcode)
-8
View File
@@ -111,12 +111,6 @@ def upload_media(noderange, media):
resource = '/noderange/{0}/media/uploads/'.format(noderange)
upargs = {'filename': filename}
noderrs = {}
if session.unixdomain:
of = open(filename, 'rb')
try:
session.add_file(filename, of.fileno(), 'rb')
except Exception:
pass
nodeurls = {}
for res in session.create(resource, upargs):
if 'created' not in res:
@@ -193,7 +187,5 @@ def main():
argparser.print_help()
sys.exit(1)
handler(noderange, media)
if __name__ == '__main__':
main()
sys.exit(exitcode)
-145
View File
@@ -1,145 +0,0 @@
#!/usr/bin/python3
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2016-2017 Lenovo
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from collections import deque
import optparse
import os
import select
import signal
import subprocess
import sys
try:
signal.signal(signal.SIGPIPE, signal.SIG_DFL)
except AttributeError:
pass
path = os.path.dirname(os.path.realpath(__file__))
path = os.path.realpath(os.path.join(path, '..', 'lib', 'python'))
if path.startswith('/opt'):
sys.path.append(path)
import confluent.client as client
import confluent.sortutil as sortutil
def run():
argparser = optparse.OptionParser(
usage="Usage: %prog [options] noderange")
argparser.add_option('-f', '-c', '--count', type='int', default=168,
help='Number of commands to run at a time')
argparser.add_option('-o', '--origname', action='store_true',
help='Use original nodename in print out even if substituted')
argparser.add_option('-s', '--substitutename',
help='Use a different name other than the nodename for ping, with {}, it is the entire name evaluated as an expression, otherwise it is used as a suffix')
# among other things, FD_SETSIZE limits. Besides, spawning too many
# processes can be unkind for the unaware on memory pressure and such...
(options, args) = argparser.parse_args()
if len(args) < 1:
argparser.print_help()
sys.exit(1)
client.check_globbing(args[0])
concurrentprocs = options.count
c = client.Command()
currprocs = 0
all = set([])
pipedesc = {}
pendingexecs = deque()
exitcode = 0
nodemap = {}
if options.substitutename:
subname = options.substitutename
if '{' not in subname:
subname = '{node}' + subname
for exp in c.create('/noderange/{0}/attributes/expression'.format(args[0]),
{'expression': subname}):
if 'error' in exp:
sys.stderr.write(exp['error'] + '\n')
exitcode |= exp.get('errorcode', 1)
ex = exp.get('databynode', ())
for node in ex:
nodemap[node] = ex[node]['value']
for rsp in c.read('/noderange/{0}/nodes/'.format(args[0])):
if 'error' in rsp:
sys.stderr.write(rsp['error'] + '\n')
exitcode |= rsp.get('errorcode', 1)
if rsp.get('item', {}).get('href', None):
node = rsp['item']['href'][:-1]
pingnode = nodemap.get(node, node)
cmdv = ['ping', '-c', '1', '-W', '1', pingnode]
if currprocs < concurrentprocs:
currprocs += 1
if options.origname:
run_cmdv(node, cmdv, all, pipedesc)
else:
run_cmdv(pingnode, cmdv, all, pipedesc)
else:
pendingexecs.append((pingnode, cmdv))
if not all or exitcode:
sys.exit(exitcode)
rdy, _, _ = select.select(all, [], [], 10)
while all:
pernodeout = {}
for r in rdy:
desc = pipedesc[r]
node = desc['node']
data = True
while data and select.select([r], [], [], 0)[0]:
data = r.readline()
if not data:
pop = desc['popen']
ret = pop.poll()
if ret is not None:
exitcode = exitcode | ret
all.discard(r)
r.close()
if desc['type'] == 'stdout':
if ret:
print('{0}: no_ping'.format(node))
else:
print('{0}: ping'.format(node))
if pendingexecs:
node, cmdv = pendingexecs.popleft()
run_cmdv(node, cmdv, all, pipedesc)
for node in sortutil.natural_sort(pernodeout):
for line in pernodeout[node]:
line = client.stringify(line)
if options.nonodeprefix:
sys.stdout.write(line)
else:
sys.stdout.write('{0}: {1}'.format(node, line))
sys.stdout.flush()
if all:
rdy, _, _ = select.select(all, [], [], 10)
sys.exit(exitcode)
def run_cmdv(node, cmdv, all, pipedesc):
nopen = subprocess.Popen(
cmdv, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
pipedesc[nopen.stdout] = {'node': node, 'popen': nopen,
'type': 'stdout'}
pipedesc[nopen.stderr] = {'node': node, 'popen': nopen,
'type': 'stderr'}
all.add(nopen.stdout)
all.add(nopen.stderr)
if __name__ == '__main__':
run()
+7 -15
View File
@@ -32,8 +32,8 @@ if path.startswith('/opt'):
import confluent.client as client
argparser = optparse.OptionParser(
usage="Usage: %prog [options] <noderange> "
"([status|on|off|shutdown|boot|reset|pdu_status|pdu_off|pdu_on])")
usage="Usage: %prog [options] noderange "
"([status|on|off|shutdown|boot|reset])")
argparser.add_option('-p', '--showprevious', dest='previous',
action='store_true', default=False,
help='Show previous power state')
@@ -51,27 +51,22 @@ except IndexError:
client.check_globbing(noderange)
setstate = None
if len(args) > 1:
setstate = args[1]
if setstate == 'softoff':
setstate = 'shutdown'
elif not args[1] in ('stat', 'state', 'status'):
setstate = args[1]
if setstate not in (None, 'on', 'off', 'shutdown', 'boot', 'reset', 'pdu_status', 'pdu_stat', 'pdu_on', 'pdu_off', 'status', 'stat', 'state'):
if setstate not in (None, 'on', 'off', 'shutdown', 'boot', 'reset'):
argparser.print_help()
sys.exit(1)
session = client.Command()
exitcode = 0
session.add_precede_key('oldstate')
powurl = 'state'
if setstate and setstate.startswith('pdu_'):
setstate = setstate.replace('pdu_', '')
powurl = 'inlets/all'
if setstate in ('status', 'state', 'stat'):
setstate = None
if options.previous:
# get previous states
prev = {}
for rsp in session.read("/noderange/{0}/power/{1}".format(noderange, powurl)):
for rsp in session.read("/noderange/{0}/power/state".format(noderange)):
# gets previous (current) states
databynode = rsp["databynode"]
@@ -82,7 +77,4 @@ if options.previous:
# add dictionary to session
session.add_precede_dict(prev)
def outhandler(node, res):
for k in res[node]:
client.cprint('{0}: {1}: {2}'.format(node, k.replace('inlet_', ''), res[node][k]))
sys.exit(session.simple_noderange_command(noderange, '/power/{0}'.format(powurl), setstate, promptover=options.maxnodes, key='state', outhandler=outhandler))
sys.exit(session.simple_noderange_command(noderange, '/power/state', setstate, promptover=options.maxnodes))
+1 -1
View File
@@ -33,7 +33,7 @@ if path.startswith('/opt'):
import confluent.client as client
argparser = optparse.OptionParser(
usage='''\n %prog <noderange>
usage='''\n %prog noderange
\n ''')
argparser.add_option('-m', '--maxnodes', type='int',
help='Specify a maximum number of '
+2 -19
View File
@@ -40,16 +40,12 @@ import confluent.sortutil as sortutil
def run():
argparser = optparse.OptionParser(
usage="Usage: %prog <file/directorylist> <noderange>:<destination>",
usage="Usage: %prog location noderange:location",
)
argparser.add_option('-m', '--maxnodes', type='int',
help='Specify a maximum number of '
'nodes to run rsync to, '
'prompting if over the threshold')
argparser.add_option('-l', '--loginname', type='str',
help='Username to use when connecting, defaults to current user.')
argparser.add_option('-s', '--substitutename',
help='Use a different name other than the nodename for rsync')
argparser.add_option('-f', '-c', '--count', type='int', default=168,
help='Number of nodes to concurrently rsync')
# among other things, FD_SETSIZE limits. Besides, spawning too many
@@ -65,20 +61,7 @@ def run():
c = client.Command()
cmdstr = ' '.join(args[:-1])
cmdstr = 'rsync -av --info=progress2 ' + cmdstr
targname = options.substitutename
if targname and '{' in targname:
targname = targname + ':'
elif targname:
targname = '{node}' + targname + ':'
else:
targname = '{node}:'
if options.loginname:
cmdstr += ' {}@'.format(options.loginname) + targname + targpath
else:
cmdstr += ' {}'.format(targname) + targpath
cmdstr += ' {node}:' + targpath
currprocs = 0
all = set([])
+3 -12
View File
@@ -35,14 +35,11 @@ if path.startswith('/opt'):
import confluent.client as client
import confluent.sortutil as sortutil
devnull = None
def run():
global devnull
devnull = open(os.devnull, 'rb')
argparser = optparse.OptionParser(
usage="Usage: %prog [options] <noderange> <command expression>",
usage="Usage: %prog [options] noderange commandexpression",
epilog="Expressions are the same as in attributes, e.g. "
"'ipmitool -H {hardwaremanagement.manager}' will be expanded.")
argparser.add_option('-f', '-c', '--count', type='int', default=168,
@@ -134,14 +131,8 @@ def run():
def run_cmdv(node, cmdv, all, pipedesc):
try:
nopen = subprocess.Popen(
cmdv, stdin=devnull, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
except OSError as e:
if e.errno == 2:
sys.stderr.write('{0}: Unable to find local executable file "{1}"'.format(node, cmdv[0]))
return
raise
nopen = subprocess.Popen(
cmdv, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
pipedesc[nopen.stdout] = {'node': node, 'popen': nopen,
'type': 'stdout'}
pipedesc[nopen.stderr] = {'node': node, 'popen': nopen,
+3 -18
View File
@@ -32,19 +32,6 @@ path = os.path.realpath(os.path.join(path, '..', 'lib', 'python'))
if path.startswith('/opt'):
sys.path.append(path)
class hybridcsv(csv.excel):
lineterminator = '\n'
def floatformat(num):
fm = u'{:.5f}'.format(num).rstrip('0')
if fm[-1:] == u'.':
return fm + u'0'
return fm
csv.register_dialect('hybrid', hybridcsv)
import confluent.client as client
sensorcollections = {
@@ -67,7 +54,7 @@ argparser.add_option('-n', '--numreadings', type='int',
argparser.add_option('-c', '--csv', action='store_true',
help='Output in CSV format')
argparser.add_option('-s', '--skipnumberless', action='store_true',
help='Do not show non-numeric sensors')
help='Output in CSV format')
(options, args) = argparser.parse_args()
repeatmode = False
if options.interval:
@@ -141,7 +128,7 @@ def sensorpass(showout=True, appendtime=False):
if sensedata['value'] is None:
showval = ''
elif isinstance(sensedata['value'], float):
showval = u' {0} '.format(floatformat(sensedata['value']))
showval = u' {0:.5f} '.format(sensedata['value'])
else:
showval = u' {0} '.format(sensedata['value'])
if sensedata['units'] not in (None, u''):
@@ -197,8 +184,6 @@ def format_csv(csvwriter, orderedsensors, resdata, showtime=True):
datum = ','.join([datum, healthstates])
else:
datum = healthstates
if isinstance(datum, float):
datum = floatformat(datum)
rowdata.append(datum)
except KeyError:
rowdata.append('N/A')
@@ -224,7 +209,7 @@ def main():
headernames.append(headername)
if options.csv:
linebyline = False
csvwriter = csv.writer(sys.stdout, dialect='hybrid')
csvwriter = csv.writer(sys.stdout)
if options.interval:
csvwriter.writerow(['time', 'node'] + headernames)
else:
+1 -1
View File
@@ -32,7 +32,7 @@ if path.startswith('/opt'):
import confluent.client as client
argparser = optparse.OptionParser(
usage='Usage: %prog [options] <noderange> [default|cd|network|setup|hd|usb|floppy]')
usage='Usage: %prog [options] noderange [default|cd|network|setup|hd]')
argparser.add_option('-b', '--bios', dest='biosmode',
action='store_true', default=False,
help='Request BIOS style boot (rather than UEFI)')
+23 -58
View File
@@ -34,31 +34,24 @@ if path.startswith('/opt'):
import confluent.client as client
import confluent.sortutil as sortutil
devnull = None
def run():
global devnull
devnull = open(os.devnull, 'rb')
argparser = optparse.OptionParser(
usage="Usage: %prog [options] noderange commandexpression",
epilog="Expressions are the same as in attributes, e.g. "
"'ipmitool -H {hardwaremanagement.manager}' will be expanded.")
argparser.add_option('-f', '-c', '--count', type='int', default=168,
help='Number of commands to run at a time')
argparser.add_option('-l', '--loginname', type='str',
help='Username to use when connecting, defaults to current user.')
argparser.add_option('-n', '--nonodeprefix', action='store_true',
help='Do not prefix output with node names')
argparser.add_option('-p', '--port', type='int', default=0,
help='Specify a custom port for ssh')
argparser.add_option('-s', '--substitutename',
help='Use a different name other than the nodename for ssh')
argparser.add_option('-x', '--noexpression', action='store_true',
help='Suppress expression expansion of command')
argparser.add_option('-m', '--maxnodes', type='int',
help='Specify a maximum number of '
'nodes to run remote ssh command to, '
'prompting if over the threshold')
'prompting if over the threshold')
# among other things, FD_SETSIZE limits. Besides, spawning too many
# processes can be unkind for the unaware on memory pressure and such...
argparser.disable_interspersed_args()
@@ -78,52 +71,25 @@ def run():
exitcode = 0
c.stop_if_noderange_over(args[0], options.maxnodes)
nodemap = {}
if options.substitutename:
subname = options.substitutename
if '{' not in subname:
subname = '{node}' + subname
for exp in c.create('/noderange/{0}/attributes/expression'.format(args[0]),
{'expression': subname}):
if 'error' in exp:
sys.stderr.write(exp['error'] + '\n')
exitcode |= exp.get('errorcode', 1)
ex = exp.get('databynode', ())
for node in ex:
nodemap[node] = ex[node]['value']
cmdparms = []
if options.noexpression:
for res in c.read('/noderange/{0}/nodes/'.format(args[0])):
if 'error' in res:
sys.stderr.write(res['error'] + '\n')
exitcode |= res.get('errorcode', 1)
break
node = res['item']['href'][:-1]
cmdparms.append((node, cmdstr))
else:
for exp in c.create('/noderange/{0}/attributes/expression'.format(args[0]),
{'expression': cmdstr}):
if 'error' in exp:
sys.stderr.write(exp['error'] + '\n')
exitcode |= exp.get('errorcode', 1)
ex = exp.get('databynode', ())
for node in ex:
cmdparms.append((node, ex[node]['value']))
for node, cmd in cmdparms:
sshnode = nodemap.get(node, node)
if not isinstance(cmd, str) and not isinstance(cmd, bytes):
cmd = cmd.encode('utf-8')
cmdv = ['ssh']
if options.port:
cmdv += ['-p', '{0}'.format(options.port)]
if options.loginname:
cmdv += ['-l', options.loginname]
cmdv += [sshnode, cmd]
if currprocs < concurrentprocs:
currprocs += 1
run_cmdv(node, cmdv, all, pipedesc)
else:
pendingexecs.append((node, cmdv))
for exp in c.create('/noderange/{0}/attributes/expression'.format(args[0]),
{'expression': cmdstr}):
if 'error' in exp:
sys.stderr.write(exp['error'] + '\n')
exitcode |= exp.get('errorcode', 1)
ex = exp.get('databynode', ())
for node in ex:
cmd = ex[node]['value']
if not isinstance(cmd, str) and not isinstance(cmd, bytes):
cmd = cmd.encode('utf-8')
if options.port:
cmdv = ['ssh', '-p', '{0}'.format(options.port), node, cmd]
else:
cmdv = ['ssh', node, cmd]
if currprocs < concurrentprocs:
currprocs += 1
run_cmdv(node, cmdv, all, pipedesc)
else:
pendingexecs.append((node, cmdv))
if not all or exitcode:
sys.exit(exitcode)
rdy, _, _ = select.select(all, [], [], 10)
@@ -163,7 +129,6 @@ def run():
if options.nonodeprefix:
sys.stdout.write(line)
else:
line = line.lstrip('\x08')
sys.stdout.write('{0}: {1}'.format(node, line))
sys.stdout.flush()
if all:
@@ -173,7 +138,7 @@ def run():
def run_cmdv(node, cmdv, all, pipedesc):
nopen = subprocess.Popen(
cmdv, stdin=devnull, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
cmdv, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
pipedesc[nopen.stdout] = {'node': node, 'popen': nopen,
'type': 'stdout'}
pipedesc[nopen.stderr] = {'node': node, 'popen': nopen,
+1 -14
View File
@@ -20,10 +20,6 @@ import os
import signal
import sys
try:
input = raw_input
except NameError:
pass
try:
signal.signal(signal.SIGPIPE, signal.SIG_DFL)
except AttributeError:
@@ -190,7 +186,7 @@ funmap = {
def main():
argparser = OptParser(
usage='Usage: %prog <noderange> [show|create|delete|diskset] [hotspare|jbod|unconfigured] [options]',
usage='Usage: %prog <noderange> [show|create|delete|diskset]',
epilog='',
)
argparser.add_option('-r', '--raidlevel', type='int',
@@ -199,8 +195,6 @@ def main():
help='Comma separated list of disks to use, or the '
'word "rest" to indicate use of all available '
'disks')
argparser.add_option('-i', '--interactive', default=False, action='store_true',
help='Confirm any potentially destructive changes interactively')
argparser.add_option('-s', '--size', type='str',
help='Comma separated list of sizes to use when '
'creating volumes. The sizes may be absolute '
@@ -236,13 +230,6 @@ def main():
except KeyError:
argparser.print_help()
sys.exit(1)
if options.interactive and operation != 'show':
showstorage(noderange, None, None)
sys.stdout.write('The above configuration will be changed by this action, do you wish to continue (y/N): ')
confirm = input()
if confirm and confirm.lower()[0] != 'y':
sys.stdout.write('Aborting\n')
sys.exit(1)
handler(noderange, options, args[2:])
+1 -1
View File
@@ -116,7 +116,7 @@ class OptParser(optparse.OptionParser):
def main():
argparser = OptParser(
usage="Usage: %prog <noderange> servicedata "
"<directory or filename>",
"<filename>",
epilog='\nservicedata will save service data to the given '
'directory. It is saved to the location on the relevant '
'management server (the confluent server if running remote, '
+13 -48
View File
@@ -16,50 +16,23 @@
# limitations under the License.
import argparse
import base64
import csv
import fcntl
import io
import numpy as np
import os
import sixel
import subprocess
import sys
try:
import sixel
class DumbWriter(sixel.SixelWriter):
def restore_position(self, output):
return
except ImportError:
pass
class DumbWriter(sixel.SixelWriter):
def restore_position(self, output):
return
def iterm_draw(data):
databuf = data.getbuffer()
datalen = len(databuf)
data = base64.b64encode(databuf).decode('utf8')
sys.stdout.write(
'\x1b]1337;File=inline=1;size={}:'.format(datalen))
sys.stdout.write(data)
sys.stdout.write('\a')
sys.stdout.write('\n')
sys.stdout.flush()
def kitty_draw(data):
data = base64.b64encode(data.getbuffer())
while data:
chunk, data = data[:4096], data[4096:]
m = 1 if data else 0
sys.stdout.write('\x1b_Ga=T,f=100,m={};'.format(m))
sys.stdout.write(chunk.decode('utf8'))
sys.stdout.write('\x1b\\')
sys.stdout.flush()
sys.stdout.write('\n')
def plot(gui, output, plotdata, bins, fmt):
def plot(gui, output, plotdata, bins):
import matplotlib as mpl
if gui and mpl.get_backend() == 'agg':
sys.stderr.write('Error: No GUI backend available and -g specified!\n')
if not gui:
mpl.use('Agg')
import matplotlib.pyplot as plt
@@ -72,15 +45,8 @@ def plot(gui, output, plotdata, bins, fmt):
tdata = io.BytesIO()
plt.savefig(tdata)
if not gui and not output:
if fmt == 'environment':
fmt = os.environ.get('CONFLUENT_IMAGE_PROTOCOL', 'kitty')
if fmt == 'sixel':
writer = DumbWriter()
writer.draw(tdata)
elif fmt == 'kitty':
kitty_draw(tdata)
elif fmt == 'iterm':
iterm_draw(tdata)
writer = DumbWriter()
writer.draw(tdata)
return n, bins
def textplot(plotdata, bins):
@@ -109,8 +75,7 @@ histogram = False
aparser = argparse.ArgumentParser(description='Quick access to common statistics')
aparser.add_argument('-c', type=int, default=0, help='Column number to analyze (default is last column)')
aparser.add_argument('-d', default=None, help='Value used to separate columns')
aparser.add_argument('-x', default=False, action='store_true', help='Output histogram in graphical format')
aparser.add_argument('-f', default='environment', help='Format for histogram output (sixel/iterm/kitty)')
aparser.add_argument('-x', default=False, action='store_true', help='Output histogram in sixel format')
aparser.add_argument('-s', default=0, help='Number of header lines to skip before processing')
aparser.add_argument('-g', default=False, action='store_true', help='Open histogram in separate graphical window')
aparser.add_argument('-o', default=None, help='Output histogram to the specified filename in PNG format')
@@ -167,7 +132,7 @@ while data:
data = list(csv.reader([data], delimiter=delimiter))[0]
n = None
if args.g or args.o or args.x:
n, bins = plot(args.g, args.o, plotdata, bins=args.b, fmt=args.f)
n, bins = plot(args.g, args.o, plotdata, bins=args.b)
if args.t:
n, bins = textplot(plotdata, bins=args.b)
print('Samples: {5} Min: {3} Median: {0} Mean: {1} Max: {4} StandardDeviation: {2} Sum: {6}'.format(np.median(plotdata), np.mean(plotdata), np.std(plotdata), np.min(plotdata), np.max(plotdata), len(plotdata), np.sum(plotdata)))
+25 -87
View File
@@ -36,22 +36,13 @@ SO_PASSCRED = 16
_attraliases = {
'bmc': 'hardwaremanagement.manager',
'bmcuser': 'secret.hardwaremanagementuser',
'switchuser': 'secret.hardwaremanagementuser',
'bmcpass': 'secret.hardwaremanagementpassword',
'switchpass': 'secret.hardwaremanagementpassword',
}
try:
getinput = raw_input
input = raw_input
except NameError:
getinput = input
class NestedDict(dict):
def __missing__(self, key):
value = self[key] = type(self)()
return value
pass
def stringify(instr):
# Normalize unicode and bytes to 'str', correcting for
@@ -118,12 +109,6 @@ def printerror(res, node=None):
exitcode = 0
if 'errorcode' in res:
exitcode = res['errorcode']
for node in res.get('databynode', {}):
exitcode = res['databynode'][node].get('errorcode', exitcode)
if 'error' in res['databynode'][node]:
sys.stderr.write('{0}: {1}\n'.format(node, res['databynode'][node]['error']))
if exitcode == 0:
exitcode = 1
if 'error' in res:
if node:
sys.stderr.write('{0}: {1}\n'.format(node, res['error']))
@@ -161,7 +146,6 @@ class Command(object):
self._prevkeyname = None
self.connection = None
self._currnoderange = None
self.unixdomain = False
if server is None:
if 'CONFLUENT_HOST' in os.environ:
self.serverloc = os.environ['CONFLUENT_HOST']
@@ -171,13 +155,11 @@ class Command(object):
self.serverloc = server
if os.path.isabs(self.serverloc) and os.path.exists(self.serverloc):
self._connect_unix()
self.unixdomain = True
elif self.serverloc == '/var/run/confluent/api.sock':
raise Exception('Confluent service is not available')
else:
self._connect_tls()
self.protversion = int(tlvdata.recv(self.connection).split(
b'--')[1].strip()[1:])
tlvdata.recv(self.connection)
authdata = tlvdata.recv(self.connection)
if authdata['authpassed'] == 1:
self.authenticated = True
@@ -188,13 +170,6 @@ class Command(object):
passphrase = os.environ['CONFLUENT_PASSPHRASE']
self.authenticate(username, passphrase)
def add_file(self, name, handle, mode):
if self.protversion < 3:
raise Exception('Not supported with connected confluent server')
if not self.unixdomain:
raise Exception('Can only add a file to a unix domain connection')
tlvdata.send(self.connection, {'filename': name, 'mode': mode}, handle)
def authenticate(self, username, password):
tlvdata.send(self.connection,
{'username': username, 'password': password})
@@ -208,7 +183,7 @@ class Command(object):
def add_precede_dict(self, dict):
self._prevdict = dict
def handle_results(self, ikey, rc, res, errnodes=None, outhandler=None):
def handle_results(self, ikey, rc, res, errnodes=None):
if 'error' in res:
if errnodes is not None:
errnodes.add(self._currnoderange)
@@ -245,12 +220,10 @@ class Command(object):
node, val, self._prevdict[node]))
else:
cprint('{0}: {1}'.format(node, val))
elif outhandler:
outhandler(node, res)
return rc
def simple_noderange_command(self, noderange, resource, input=None,
key=None, errnodes=None, promptover=None, outhandler=None, **kwargs):
key=None, errnodes=None, promptover=None, **kwargs):
try:
self._currnoderange = noderange
rc = 0
@@ -264,13 +237,13 @@ class Command(object):
if input is None:
for res in self.read('/noderange/{0}/{1}'.format(
noderange, resource)):
rc = self.handle_results(ikey, rc, res, errnodes, outhandler)
rc = self.handle_results(ikey, rc, res, errnodes)
else:
self.stop_if_noderange_over(noderange, promptover)
kwargs[ikey] = input
for res in self.update('/noderange/{0}/{1}'.format(
noderange, resource), kwargs):
rc = self.handle_results(ikey, rc, res, errnodes, outhandler)
rc = self.handle_results(ikey, rc, res, errnodes)
self._currnoderange = None
return rc
except KeyboardInterrupt:
@@ -286,9 +259,9 @@ class Command(object):
nodename = list(self.read(
'/noderange/{0}/nodes/'.format(noderange)))[0].get('item', {}).get('href', None)
nodename = nodename[:-1]
p = getinput('Command is about to affect node {0}, continue (y/n)? '.format(nodename))
p = input('Command is about to affect node {0}, continue (y/n)? '.format(nodename))
else:
p = getinput('Command is about to affect {0} nodes, continue (y/n)? '.format(nsize))
p = input('Command is about to affect {0} nodes, continue (y/n)? '.format(nsize))
if p.lower() != 'y':
sys.stderr.write('Aborting at user request\n')
sys.exit(1)
@@ -403,7 +376,7 @@ class Command(object):
if fingerprint == khf[hostid]:
return
else:
replace = getinput(
replace = input(
"MISMATCHED CERTIFICATE DATA, ACCEPT NEW? (y/n):")
if replace not in ('y', 'Y'):
raise Exception("BAD CERTIFICATE")
@@ -436,7 +409,7 @@ def send_request(operation, path, server, parameters=None):
result = tlvdata.recv(server)
def attrrequested(attr, attrlist, seenattributes, node=None):
def attrrequested(attr, attrlist, seenattributes):
for candidate in attrlist:
truename = candidate
if candidate.startswith('hm'):
@@ -444,16 +417,10 @@ def attrrequested(attr, attrlist, seenattributes, node=None):
if candidate in _attraliases:
candidate = _attraliases[candidate]
if fnmatch.fnmatch(attr.lower(), candidate.lower()):
if node is None:
seenattributes.add(truename)
else:
seenattributes[node][truename] = True
seenattributes.add(truename)
return True
elif attr.lower().startswith(candidate.lower() + '.'):
if node is None:
seenattributes.add(truename)
else:
seenattributes[node][truename] = 1
seenattributes.add(truename)
return True
return False
@@ -469,15 +436,13 @@ def _sort_attrib(k):
def print_attrib_path(path, session, requestargs, options, rename=None, attrprefix=None):
exitcode = 0
seenattributes = NestedDict()
allnodes = set([])
seenattributes = set([])
for res in session.read(path):
if 'error' in res:
sys.stderr.write(res['error'] + '\n')
exitcode = 1
continue
for node in sorted(res['databynode']):
allnodes.add(node)
for attr, val in sorted(res['databynode'][node].items(), key=_sort_attrib):
if attr == 'error':
sys.stderr.write('{0}: Error: {1}\n'.format(node, val))
@@ -485,7 +450,7 @@ def print_attrib_path(path, session, requestargs, options, rename=None, attrpref
if attr == 'errorcode':
exitcode |= val
continue
seenattributes[node][attr] = True
seenattributes.add(attr)
if rename:
printattr = rename.get(attr, attr)
else:
@@ -493,14 +458,14 @@ def print_attrib_path(path, session, requestargs, options, rename=None, attrpref
if attrprefix:
printattr = attrprefix + printattr
currattr = res['databynode'][node][attr]
if show_attr(attr, requestargs, seenattributes, options, node):
if show_attr(attr, requestargs, seenattributes, options):
if 'value' in currattr:
if currattr['value'] is not None:
val = currattr['value']
if isinstance(val, list):
val = ','.join(val)
attrout = '{0}: {1}: {2}'.format(
node, printattr, val).strip()
node, printattr, val)
else:
attrout = '{0}: {1}:'.format(node, printattr)
elif 'isset' in currattr:
@@ -509,7 +474,7 @@ def print_attrib_path(path, session, requestargs, options, rename=None, attrpref
printattr)
else:
attrout = '{0}: {1}:'.format(node, printattr)
elif isinstance(currattr, dict) and 'broken' in currattr:
elif 'broken' in currattr:
attrout = '{0}: {1}: *ERROR* BROKEN EXPRESSION: ' \
'{2}'.format(node, printattr,
currattr['broken'])
@@ -526,7 +491,7 @@ def print_attrib_path(path, session, requestargs, options, rename=None, attrpref
blame = options.blame
except AttributeError:
blame = False
if blame or (isinstance(currattr, dict) and 'broken' in currattr):
if blame or 'broken' in currattr:
blamedata = []
if 'inheritedfrom' in currattr:
blamedata.append('inherited from group {0}'.format(
@@ -574,39 +539,23 @@ def print_attrib_path(path, session, requestargs, options, rename=None, attrpref
except TypeError:
pass
cprint(attrout)
somematched = set([])
printmissing = set([])
badnodes = NestedDict()
if not exitcode:
if requestargs:
for attr in requestargs:
for node in allnodes:
if attr in seenattributes[node]:
somematched.add(attr)
else:
badnodes[node][attr] = True
exitcode = 1
for node in sortutil.natural_sort(badnodes):
for attr in badnodes[node]:
if attr in somematched:
sys.stderr.write(
'Error: {0} matches no valid value for {1}\n'.format(
attr, node))
else:
printmissing.add(attr)
for missing in printmissing:
sys.stderr.write('Error: {0} not a valid attribute\n'.format(missing))
if attr not in seenattributes:
sys.stderr.write('Error: {0} not a valid attribute\n'.format(attr))
exitcode = 1
return exitcode
def show_attr(attr, requestargs, seenattributes, options, node):
def show_attr(attr, requestargs, seenattributes, options):
try:
reverse = options.exclude
except AttributeError:
reverse = False
if requestargs is None or requestargs == []:
return True
processattr = attrrequested(attr, requestargs, seenattributes, node)
processattr = attrrequested(attr, requestargs, seenattributes)
if reverse:
processattr = not processattr
return processattr
@@ -635,7 +584,7 @@ def printgroupattributes(session, requestargs, showtype, nodetype, noderange, op
attrout = '{0}: {1}: ********'.format(noderange, attr)
else:
attrout = '{0}: {1}:'.format(noderange, attr)
elif isinstance(currattr, dict) and 'broken' in currattr:
elif 'broken' in currattr:
attrout = '{0}: {1}: *ERROR* BROKEN EXPRESSION: ' \
'{2}'.format(noderange, attr,
currattr['broken'])
@@ -668,9 +617,6 @@ def updateattrib(session, updateargs, nodetype, noderange, options, dictassign=N
for attrib in updateargs[1:]:
keydata[attrib] = None
for res in session.update(targpath, keydata):
for node in res.get('databynode', {}):
for warnmsg in res['databynode'][node].get('_warnings', []):
sys.stderr.write('Warning: ' + warnmsg + '\n')
if 'error' in res:
if 'errorcode' in res:
exitcode = res['errorcode']
@@ -705,14 +651,6 @@ def updateattrib(session, updateargs, nodetype, noderange, options, dictassign=N
noderange, 'attributes/all', dictassign[key], key)
else:
if "=" in updateargs[1]:
update_ready = True
for arg in updateargs[1:]:
if not '=' in arg:
update_ready = False
exitcode = 1
if not update_ready:
sys.stderr.write('Error: {0} Can not set and read at the same time!\n'.format(str(updateargs[1:])))
sys.exit(exitcode)
try:
for val in updateargs[1:]:
val = val.split('=', 1)
-342
View File
@@ -1,342 +0,0 @@
#!/usr/bin/python3
import collections
import os
import struct
import sys
import time
import fcntl
import select
import termios
import tty
def writeout(data):
if isinstance(data, str) and not isinstance(data, bytes):
data = data.encode('utf8')
try:
if hasattr(sys.stdout, 'buffer'):
sys.stdout.buffer.write(data)
else:
sys.stdout.write(data)
except IOError:
time.sleep(0.1)
pass
class LogReplay(object):
def __init__(self, logfile, cblfile):
self.bin = open(cblfile, 'rb')
self.txt = open(logfile, 'rb')
self.cleardata = []
self.clearidx = 0
self.pendingdata = collections.deque([])
self.priordata = collections.deque([])
self.laststamp = None
self.needclear = False
self.lasttxt = ''
def _rewind(self, datasize=None):
curroffset = self.bin.tell() - 16
if self.cleardata and self.clearidx > 1:
self.clearidx -= 1
priordata = self.cleardata[self.clearidx - 1]
return curroffset, priordata
self.cleardata = []
self.clearidx = 0
newoffset = curroffset - 32
if newoffset < 0: #TODO: Follow a log roll
newoffset = 0
if datasize:
while datasize > 0 and newoffset > 0:
self.bin.seek(newoffset)
tmprec = self.bin.read(16)
newoffset -= 32
tmprec = struct.unpack('!BBIHIBBH', tmprec)
if tmprec[1] == 2:
datasize -= tmprec[3]
if newoffset >= 0:
self.bin.seek(newoffset)
return curroffset, None
def debuginfo(self):
return '{0}, {1}'.format(self.bin.tell(), self.clearidx)
def get_output(self, reverse=False):
endoffset = None
output = b''
if reverse: # Forget the uncommited future, if present
output += b'\x1b[2J\x1b[H'
endoffset, priordata = self._rewind(4096)
if priordata is not None:
return priordata, 1
elif self.needclear:
output += b'\x1b[2J\x1b[H'
self.needclear = False
if self.cleardata and self.clearidx < len(self.cleardata):
datachunk = self.cleardata[self.clearidx]
self.clearidx += 1
return datachunk, 1
self.cleardata = []
self.clearidx = 0
while (not reverse) or (self.bin.tell() < endoffset):
record = self.bin.read(16)
if not record:
return b'', 0
record = struct.unpack('!BBIHIBBH', record)
if record[0] > 16:
# Unsupported record, skip
self.bin.seek(record[0] - 16, 1)
continue
type = record[1]
offset = record[2]
size = record[3]
evtdata = record[5]
auxdata = record[6]
if type == 3:
#TODO: provide data for status bar
continue
elif type == 2:
self.laststamp = record[4]
self.txt.seek(offset)
txtout = self.txt.read(size)
if reverse and self.bin.tell() < endoffset:
output += txtout
continue
self.paginate(txtout)
if self.cleardata:
self.clearidx = 0
if not self.cleardata[0]:
self.cleardata = self.cleardata[1:]
if self.cleardata:
if reverse:
output = self.cleardata[-1]
self.clearidx = len(self.cleardata)
else:
output += self.cleardata[0]
self.clearidx = 1
else:
output += txtout
break
if endoffset is not None and endoffset >= 0:
self.bin.seek(endoffset)
self.lasttxt = output
return output, 1
def paginate(self, txtout):
cleardata = [txtout]
nextcleardata = []
for sep in (b'\x1b[2J', b'\x1b[H\x1b[J'):
replacementcleardata = []
for txtout in cleardata:
nextcleardata = txtout.split(sep)
if len(nextcleardata) > 1:
for idx in range(1, len(nextcleardata)):
nextcleardata[idx] = sep + nextcleardata[idx]
replacementcleardata.extend(nextcleardata)
nextcleardata = []
cleardata = replacementcleardata
if len(cleardata) > 1:
self.cleardata = cleardata
return
def begin(self):
self.needclear = True
self.bin.seek(0)
def end(self):
self.bin.seek(0, 2)
def search(self, searchstr, skipfirst=False):
overlap = len(searchstr)
firstoffset = self.bin.tell()
lastoffset = firstoffset
output = self.lasttxt
while searchstr not in output:
output, delay = self.get_output()
if not self.cleardata and lastoffset == self.bin.tell():
self.bin.seek(firstoffset)
return b'', 0
lastoffset = self.bin.tell()
if skipfirst:
output, delay = self.get_output()
while searchstr not in output:
output, delay = self.get_output()
if not self.cleardata and lastoffset == self.bin.tell():
self.bin.seek(firstoffset)
return b'', 0
lastoffset = self.bin.tell()
clear = b'\x1b[2J\x1b[H'
txtchunks = self.lasttxt.split(searchstr)
output = clear + txtchunks[0]
txtchunks = txtchunks[1:]
while txtchunks:
output += b'\x1b[7m' + searchstr + b'\x1b[27m' + txtchunks[0]
txtchunks = txtchunks[1:]
return output, 0
def _replay_to_console(txtfile, binfile):
replay = LogReplay(txtfile, binfile)
oldtcattr = termios.tcgetattr(sys.stdin.fileno())
tty.setraw(sys.stdin.fileno())
currfl = fcntl.fcntl(sys.stdin.fileno(), fcntl.F_GETFL)
fcntl.fcntl(sys.stdin.fileno(), fcntl.F_SETFL, currfl | os.O_NONBLOCK)
reverse = False
skipnext = False
quitit = False
searchstr = None
writeout('\x1b[2J\x1b[;H')
prepend = ''
try:
while not quitit:
if not skipnext:
newdata, delay = replay.get_output(reverse)
skipnext = False
reverse = False
if newdata:
if prepend:
newdata = prepend + newdata
prepend = b''
writeout(newdata)
newdata = ''
writeout('\x1b]0;[Time: {0}]\x07'.format(
time.strftime('%m/%d %H:%M:%S', time.localtime(replay.laststamp))))
try:
if hasattr(sys.stdout, 'buffer'):
sys.stdout.buffer.flush()
else:
sys.stdout.flush()
except IOError:
pass
while True:
select.select((sys.stdin,), (), (), 86400)
myinput = sys.stdin.read()
if myinput.startswith('\x1b[C') or myinput.startswith('\x1bOC') or myinput == '\r': # right
break
elif myinput.startswith('\x1b[D') or myinput.startswith('\x1bOD') or myinput == 'y': # left
writeout('\x1b[2J\x1b[;H')
reverse = True
break
elif myinput == 'G' or myinput.startswith('\x1b[F'):
replay.end()
reverse = True
break
elif myinput == 'g' or myinput.startswith('\x1b[H'):
replay.begin()
break
elif myinput.lower() == 'q' or myinput == '\x03':
quitit = True
break
elif myinput.lower() == 'd':
writeout('\x1b];{0}\x07'.format(replay.debuginfo()))
if hasattr(sys.stdout, 'buffer'):
sys.stdout.buffer.flush()
else:
sys.stdout.flush()
elif myinput.lower() == '/':
sys.stdout.write('\x1b7\x1b[99999;0H\x1b[2K')
searchstr = ''
nxtchr = '/'
while '\r' not in searchstr:
sys.stdout.write(nxtchr)
sys.stdout.flush()
select.select((sys.stdin,), (), (), 86400)
nxtchr = sys.stdin.read(1)
if nxtchr in ('\x08', '\x7f'):
searchstr = searchstr[:-1]
nxtchr = '\x08 \x08'
else:
searchstr += nxtchr
if not isinstance(searchstr, bytes):
searchstr = searchstr.encode('utf8')
searchstr = searchstr[:-1]
newdata, delay = replay.search(searchstr)
if not newdata:
sys.stdout.write('\x1b[1K\rNo match found')
prepend = b'\x1b[1K\x1b8'
else:
sys.stdout.write('\x1b[1K\x1b8')
sys.stdout.flush()
skipnext = True
break
elif myinput.lower() == 'n' and searchstr:
newdata, delay = replay.search(searchstr, True)
if not newdata:
sys.stdout.write('\x1b7\x1b[99999;0H\x1b[2KNo more matches found')
sys.stdout.flush()
prepend = b'\x1b[1K\x1b8'
skipnext = True
break
else:
pass # print(repr(myinput))
except Exception:
currfl = fcntl.fcntl(sys.stdin.fileno(), fcntl.F_GETFL)
fcntl.fcntl(sys.stdin.fileno(), fcntl.F_SETFL, currfl ^ os.O_NONBLOCK)
termios.tcsetattr(sys.stdin.fileno(), termios.TCSANOW, oldtcattr)
writeout('\x1b[m\x1b[?25h\n')
raise
currfl = fcntl.fcntl(sys.stdin.fileno(), fcntl.F_GETFL)
fcntl.fcntl(sys.stdin.fileno(), fcntl.F_SETFL, currfl ^ os.O_NONBLOCK)
termios.tcsetattr(sys.stdin.fileno(), termios.TCSANOW, oldtcattr)
writeout('\x1b[m\x1b[?25h\n')
def replay_to_console(txtfile):
if os.path.exists(txtfile + '.cbl'):
binfile = txtfile + '.cbl'
elif '.' not in txtfile:
if '/' not in txtfile:
txtfile = os.getcwd() + '/' + txtfile
sys.stderr.write('Unable to locate cbl file: "{0}"\n'.format(txtfile + '.cbl'))
sys.exit(1)
else:
fileparts = txtfile.split('.')
prefix = '.'.join(fileparts[:-1])
binfile = prefix + '.cbl.' + fileparts[-1]
if not os.path.exists(binfile):
sys.stderr.write('Unable to locate cbl file: "{0}"\n'.format(binfile))
sys.exit(1)
_replay_to_console(txtfile, binfile)
def dump_to_console(txtfile):
if os.path.exists(txtfile + '.cbl'):
binfile = txtfile + '.cbl'
elif '.' not in txtfile:
if '/' not in txtfile:
txtfile = os.getcwd() + '/' + txtfile
sys.stderr.write('Unable to locate cbl file: "{0}"\n'.format(txtfile + '.cbl'))
sys.exit(1)
else:
fileparts = txtfile.split('.')
prefix = '.'.join(fileparts[:-1])
binfile = prefix + '.cbl.' + fileparts[-1]
if not os.path.exists(binfile):
sys.stderr.write('Unable to locate cbl file: "{0}"\n'.format(binfile))
sys.exit(1)
replay = LogReplay(txtfile, binfile)
quitit = False
writeout('\x1b[2J\x1b[;H')
prepend = ''
try:
while not quitit:
newdata, delay = replay.get_output(False)
if newdata:
if prepend:
newdata = prepend + newdata
prepend = b''
writeout(newdata.replace(b'\r\n', '\r\n[ {0} ] '.format(time.strftime('%m/%d %H:%M:%S', time.localtime(replay.laststamp))).encode('utf8')))
newdata = ''
try:
if hasattr(sys.stdout, 'buffer'):
sys.stdout.buffer.flush()
else:
sys.stdout.flush()
except IOError:
pass
else:
break
except Exception:
writeout('\x1b[m\x1b[?25h\n')
raise
writeout('\x1b[m\x1b[?25h\n')
if __name__ == '__main__':
replay_to_console(sys.argv[1])
+18 -58
View File
@@ -17,10 +17,9 @@ import sys
import struct
import termios
def get_screengeom():
# returns height in cells, width in cells, width in pixels, height in pixels
return struct.unpack('hhhh', fcntl.ioctl(sys.stdout, termios.TIOCGWINSZ,
b'........'))
def get_screenwidth():
return struct.unpack('hh', fcntl.ioctl(sys.stdout, termios.TIOCGWINSZ,
b'....'))[1]
class ScreenPrinter(object):
def __init__(self, noderange, client, textlen=4):
@@ -30,9 +29,6 @@ class ScreenPrinter(object):
self.client = client
self.nodeoutput = {}
self.nodelist = []
self.nodepos = {}
self.lastrows = 0
self.fieldchanged = False
maxlen = 0
for ans in self.client.read('/noderange/{0}/nodes/'.format(noderange)):
if 'error' in ans:
@@ -48,68 +44,33 @@ class ScreenPrinter(object):
self.fieldwidth = maxlen + textlen + 1 # 1 for column
def set_output(self, node, text):
if self.nodeoutput.get(node, None) == text:
if self.nodeoutput[node] == text:
return
self.nodeoutput[node] = text
if len(text) >= self.textlen:
self.textlen = len(text) + 1
self.fieldwidth = self.textlen + self.nodenamelen + 1
self.fieldchanged = True
self.drawscreen(node)
self.drawscreen()
def drawscreen(self, node=None):
def drawscreen(self):
if self.squeeze:
currheight, currwidth, _, _ = get_screengeom()
currheight -= 2
if currheight < 1:
currheight = 1
currwidth = get_screenwidth()
numfields = currwidth // self.fieldwidth
fieldformat = '{{0:>{0}}}:{{1:{1}}}'.format(self.nodenamelen,
self.textlen)
#sys.stdout.write('\x1b[2J\x1b[;H') # clear screen
if len(self.nodelist) < (numfields * currheight):
numfields = len(self.nodelist) // currheight + 1
sys.stdout.write('\x1b[2J\x1b[;H') # clear screen
else:
numfields = 1
fieldformat = '{0}: {1}'
if self.squeeze:
columns = [self.nodelist[x:x+currheight] for x in range(0, len(self.nodelist), currheight)]
if self.lastrows:
sys.stdout.write('\x1b[{0}A'.format(self.lastrows))
if node and self.lastrows == len(columns[0]) and not self.fieldchanged:
targline, targcol = self.nodepos[node]
if targline:
sys.stdout.write('\x1b[{0}B'.format(targline))
if targcol:
sys.stdout.write('\x1b[{0}C'.format(targcol))
sys.stdout.write(fieldformat.format(node, self.nodeoutput[node]))
sys.stdout.write('\r\x1b[{0}B'.format(self.lastrows - targline))
else:
self.lastrows = 0
self.fieldchanged = False
column = 0
for currow in range(0, len(columns[0])):
sys.stdout.write('\x1b[2K')
for col in columns:
try:
node = col[currow]
self.nodepos[node] = (currow, column)
sys.stdout.write(fieldformat.format(node, self.nodeoutput[node]))
column += len(fieldformat.format(node, self.nodeoutput[node]))
except IndexError:
break
sys.stdout.write('\n')
column = 0
self.lastrows += 1
sys.stdout.write('\x1b[J')
else:
if node:
nodes = [node]
else:
nodes = self.nodelist
for node in nodes:
sys.stdout.write(fieldformat.format(node, self.nodeoutput[node]))
currfields = 0
for node in self.nodelist:
if currfields >= numfields:
sys.stdout.write('\n')
currfields = 1
else:
currfields += 1
sys.stdout.write(fieldformat.format(node, self.nodeoutput[node]))
sys.stdout.write('\n')
sys.stdout.flush()
@@ -119,9 +80,8 @@ class ScreenPrinter(object):
if __name__ == '__main__':
import confluent.client as client
c = client.Command()
p = ScreenPrinter('d1-d12', c)
p.set_output('d3', 'Upload: 67%')
p.set_output('d7', 'Upload: 67%')
p = ScreenPrinter('n1-n13', c)
p.set_output('n3', 'Upload: 67%')
+4 -7
View File
@@ -53,16 +53,13 @@ class TermHandler(object):
connection = None
try:
connection, address = self.socket.accept()
connection.sendall(b"confetty control v1--\n")
connection.sendall("confetty control v1--\n")
cmd = connection.recv(8)
if b'GETWINID' == cmd:
winid = os.environ['WINDOWID']
if not isinstance(winid, bytes):
winid = winid.encode('utf8')
connection.sendall(winid)
if 'GETWINID' == cmd:
connection.sendall(os.environ['WINDOWID'])
connection.close()
except BaseException:
pass
finally:
if connection is not None:
connection.close()
connection.close()
+4 -15
View File
@@ -98,24 +98,17 @@ class GroupedData(object):
self.byoutput = {}
self.header = {}
self.client = confluentconnection
self.detectedpad = None
def generate_byoutput(self):
self.byoutput = {}
thepad = self.detectedpad if self.detectedpad else ''
for n in self.bynode:
output = ''
for ln in self.bynode[n]:
output += ln.replace(thepad, '', 1) + '\n'
output = '\n'.join(self.bynode[n])
if output not in self.byoutput:
self.byoutput[output] = set([n])
else:
self.byoutput[output].add(n)
def add_line(self, node, line):
wspc = re.search(r'^\s*', line).group()
if self.detectedpad is None or len(wspc) < len(self.detectedpad):
self.detectedpad = wspc
if node not in self.bynode:
self.bynode[node] = [line]
else:
@@ -171,9 +164,7 @@ class GroupedData(object):
self.byoutput[outdata])))
currout += '\n====================================\n'
currout += outdata
if currout[-1] != '\n':
currout += '\n'
currout += '\n'
currout += '\n\n'
output.write(currout)
output.flush()
@@ -213,9 +204,7 @@ class GroupedData(object):
else:
currout += '\n'.join(colordiff(modaloutput.split('\n'),
outdata.split('\n')))
if currout[-1] != '\n':
currout += '\n'
currout += '\n'
currout += '\n\n'
if reverse:
revoutput.append(currout)
else:
@@ -230,4 +219,4 @@ if __name__ == '__main__':
if not line:
continue
groupoutput.add_line(*line.split(': ', 1))
groupoutput.print_deviants()
groupoutput.print_deviants()
+1 -1
View File
@@ -17,4 +17,4 @@
class Types(object):
text, json, filehandle = range(3)
text, json = range(2)
+10 -138
View File
@@ -15,19 +15,9 @@
# See the License for the specific language governing permissions and
# limitations under the License.
import array
import ctypes
import ctypes.util
import confluent.tlv as tlv
try:
import eventlet.green.socket as socket
import eventlet.green.select as select
except ImportError:
import socket
import select
from datetime import datetime
import json
import os
import struct
try:
@@ -40,72 +30,6 @@ try:
except NameError:
pass
class iovec(ctypes.Structure): # from uio.h
_fields_ = [('iov_base', ctypes.c_void_p),
('iov_len', ctypes.c_size_t)]
iovec_ptr = ctypes.POINTER(iovec)
class cmsghdr(ctypes.Structure): # also from bits/socket.h
_fields_ = [('cmsg_len', ctypes.c_size_t),
('cmsg_level', ctypes.c_int),
('cmsg_type', ctypes.c_int)]
@classmethod
def init_data(cls, cmsg_len, cmsg_level, cmsg_type, cmsg_data):
Data = ctypes.c_ubyte * ctypes.sizeof(cmsg_data)
class _flexhdr(ctypes.Structure):
_fields_ = cls._fields_ + [('cmsg_data', Data)]
datab = Data(*bytearray(cmsg_data))
return _flexhdr(cmsg_len=cmsg_len, cmsg_level=cmsg_level,
cmsg_type=cmsg_type, cmsg_data=datab)
def CMSG_LEN(length):
sizeof_cmshdr = ctypes.sizeof(cmsghdr)
return ctypes.c_size_t(CMSG_ALIGN(sizeof_cmshdr).value + length)
SCM_RIGHTS = 1
class msghdr(ctypes.Structure): # from bits/socket.h
_fields_ = [('msg_name', ctypes.c_void_p),
('msg_namelen', ctypes.c_uint),
('msg_iov', ctypes.POINTER(iovec)),
('msg_iovlen', ctypes.c_size_t),
('msg_control', ctypes.c_void_p),
('msg_controllen', ctypes.c_size_t),
('msg_flags', ctypes.c_int)]
def CMSG_ALIGN(length): # bits/socket.h
ret = (length + ctypes.sizeof(ctypes.c_size_t) - 1
& ~(ctypes.sizeof(ctypes.c_size_t) - 1))
return ctypes.c_size_t(ret)
def CMSG_SPACE(length): # bits/socket.h
ret = CMSG_ALIGN(length).value + CMSG_ALIGN(ctypes.sizeof(cmsghdr)).value
return ctypes.c_size_t(ret)
class ClientFile(object):
def __init__(self, name, mode, fd):
self.fileobject = os.fdopen(fd, mode)
self.filename = name
libc = ctypes.CDLL(ctypes.util.find_library('c'))
recvmsg = libc.recvmsg
recvmsg.argtypes = [ctypes.c_int, ctypes.POINTER(msghdr), ctypes.c_int]
recvmsg.restype = ctypes.c_int
sendmsg = libc.sendmsg
sendmsg.argtypes = [ctypes.c_int, ctypes.POINTER(msghdr), ctypes.c_int]
sendmsg.restype = ctypes.c_size_t
def decodestr(value):
ret = None
try:
@@ -141,7 +65,7 @@ def _unicode_list(currlist):
_unicode_list(currlist[i])
def send(handle, data, filehandle=None):
def send(handle, data):
if isinstance(data, unicode):
try:
data = data.encode('utf-8')
@@ -169,27 +93,9 @@ def send(handle, data, filehandle=None):
if tl > 16777215:
raise Exception("JSON data exceeds protocol limits")
# xor in the type (0b1 << 24)
if filehandle is None:
tl |= 16777216
handle.sendall(struct.pack("!I", tl))
handle.sendall(sdata)
else:
tl |= (2 << 24)
handle.sendall(struct.pack("!I", tl))
cdtype = ctypes.c_ubyte * len(sdata)
cdata = cdtype.from_buffer(bytearray(sdata))
ciov = iovec(iov_base=ctypes.addressof(cdata),
iov_len=ctypes.c_size_t(ctypes.sizeof(cdata)))
fd = ctypes.c_int(filehandle)
cmh = cmsghdr.init_data(
cmsg_len=CMSG_LEN(
ctypes.sizeof(fd)), cmsg_level=socket.SOL_SOCKET,
cmsg_type=SCM_RIGHTS, cmsg_data=fd)
mh = msghdr(msg_name=None, msg_len=0, msg_iov=iovec_ptr(ciov),
msg_iovlen=1, msg_control=ctypes.addressof(cmh),
msg_controllen=ctypes.c_size_t(ctypes.sizeof(cmh)))
sendmsg(handle.fileno(), mh, 0)
tl |= 16777216
handle.sendall(struct.pack("!I", tl))
handle.sendall(sdata)
def recvall(handle, size):
rd = handle.recv(size)
@@ -219,46 +125,12 @@ def recv(handle):
datatype = (tl & 2130706432) >> 24 # grab 7 bits from near beginning
if dlen == 0:
return None
if datatype == tlv.Types.filehandle:
filehandles = array.array('i')
rawbuffer = bytearray(2048)
pkttype = ctypes.c_ubyte * 2048
data = pkttype.from_buffer(rawbuffer)
cmsgsize = CMSG_SPACE(ctypes.sizeof(ctypes.c_int)).value
cmsgarr = bytearray(cmsgsize)
cmtype = ctypes.c_ubyte * cmsgsize
cmsg = cmtype.from_buffer(cmsgarr)
cmsg.cmsg_level = socket.SOL_SOCKET
cmsg.cmsg_type = SCM_RIGHTS
cmsg.cmsg_len = CMSG_LEN(ctypes.sizeof(ctypes.c_int))
iov = iovec()
iov.iov_base = ctypes.addressof(data)
iov.iov_len = 2048
msg = msghdr()
msg.msg_iov = ctypes.pointer(iov)
msg.msg_iovlen = 1
msg.msg_control = ctypes.addressof(cmsg)
msg.msg_controllen = ctypes.sizeof(cmsg)
select.select([handle], [], [])
i = recvmsg(handle.fileno(), ctypes.pointer(msg), 0)
cdata = cmsgarr[CMSG_LEN(0).value:]
data = rawbuffer[:i]
if cmsg.cmsg_level == socket.SOL_SOCKET and cmsg.cmsg_type == SCM_RIGHTS:
try:
filehandles.fromstring(bytes(
cdata[:len(cdata) - len(cdata) % filehandles.itemsize]))
except AttributeError:
filehandles.frombytes(bytes(
cdata[:len(cdata) - len(cdata) % filehandles.itemsize]))
data = json.loads(bytes(data))
return ClientFile(data['filename'], data['mode'], filehandles[0])
else:
data = handle.recv(dlen)
while len(data) < dlen:
ndata = handle.recv(dlen - len(data))
if not ndata:
raise Exception("Error reading data")
data += ndata
data = handle.recv(dlen)
while len(data) < dlen:
ndata = handle.recv(dlen - len(data))
if not ndata:
raise Exception("Error reading data")
data += ndata
if datatype == tlv.Types.text:
return data
elif datatype == tlv.Types.json:
+11 -18
View File
@@ -1,23 +1,19 @@
%define name confluent_client
%define version #VERSION#
%define fversion %{lua:
sv, _ = string.gsub("#VERSION#", "[~+]", "-")
print(sv)
}
%define release 1
Summary: Client libraries and utilities for confluent
Name: %{name}
Version: %{version}
Release: %{release}
Source0: %{name}-%{fversion}.tar.gz
License: Apache2
Source0: %{name}-%{version}.tar.gz
License: UNKNOWN
Group: Development/Libraries
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-buildroot
Prefix: %{_prefix}
BuildArch: noarch
Vendor: Lenovo
Url: http://github.com/lenovo/confluent
Vendor: Jarrod Johnson <jjohnson2@lenovo.com>
Url: http://xcat.sf.net/
Obsoletes: confluent_common
%description
@@ -25,20 +21,20 @@ This package enables python development and command line access to
a confluent server.
%prep
%setup -n %{name}-%{fversion}
%setup -n %{name}-%{version} -n %{name}-%{version}
%build
%if "%{dist}" == ".el7"
python2 setup.py build
%else
%if "%{dist}" == ".el8"
python3 setup.py build
%else
python2 setup.py build
%endif
%install
%if "%{dist}" == ".el7"
python2 setup.py install --single-version-externally-managed -O1 --root=$RPM_BUILD_ROOT --record=INSTALLED_FILES --install-scripts=/opt/confluent/bin --install-purelib=/opt/confluent/lib/python
%else
%if "%{dist}" == ".el8"
python3 setup.py install --single-version-externally-managed -O1 --root=$RPM_BUILD_ROOT --record=INSTALLED_FILES --install-scripts=/opt/confluent/bin --install-purelib=/opt/confluent/lib/python
%else
python2 setup.py install --single-version-externally-managed -O1 --root=$RPM_BUILD_ROOT --record=INSTALLED_FILES --install-scripts=/opt/confluent/bin --install-purelib=/opt/confluent/lib/python
%endif
@@ -46,7 +42,4 @@ python3 setup.py install --single-version-externally-managed -O1 --root=$RPM_BUI
rm -rf $RPM_BUILD_ROOT
%files -f INSTALLED_FILES
%license /opt/confluent/share/licenses/confluent_client/LICENSE
%license /opt/confluent/share/licenses/confluent_client/COPYRIGHT
%defattr(-,root,root)
/opt/confluent
+7 -111
View File
@@ -19,7 +19,6 @@ alias nodebmcreset='CURRENT_CMDLINE=$(HISTTIMEFORMAT= builtin history 1); export
alias nodeboot='CURRENT_CMDLINE=$(HISTTIMEFORMAT= builtin history 1); export CURRENT_CMDLINE; nodeboot'
alias nodeconfig='CURRENT_CMDLINE=$(HISTTIMEFORMAT= builtin history 1); export CURRENT_CMDLINE; nodeconfig'
alias nodeconsole='CURRENT_CMDLINE=$(HISTTIMEFORMAT= builtin history 1); export CURRENT_CMDLINE; nodeconsole'
alias nodedeploy='CURRENT_CMDLINE=$(HISTTIMEFORMAT= builtin history 1); export CURRENT_CMDLINE; nodedeploy'
alias nodedefine='CURRENT_CMDLINE=$(HISTTIMEFORMAT= builtin history 1); export CURRENT_CMDLINE; nodedefine'
alias nodeeventlog='CURRENT_CMDLINE=$(HISTTIMEFORMAT= builtin history 1); export CURRENT_CMDLINE; nodeeventlog'
alias nodefirmware='CURRENT_CMDLINE=$(HISTTIMEFORMAT= builtin history 1); export CURRENT_CMDLINE; nodefirmware'
@@ -37,7 +36,7 @@ alias nodereseat='CURRENT_CMDLINE=$(HISTTIMEFORMAT= builtin history 1); export C
alias noderun='CURRENT_CMDLINE=$(HISTTIMEFORMAT= builtin history 1); export CURRENT_CMDLINE; noderun'
alias nodesensors='CURRENT_CMDLINE=$(HISTTIMEFORMAT= builtin history 1); export CURRENT_CMDLINE; nodesensors'
alias nodesetboot='CURRENT_CMDLINE=$(HISTTIMEFORMAT= builtin history 1); export CURRENT_CMDLINE; nodesetboot'
alias nodestorage='CURRENT_CMDLINE=$(HISTTIMEFORMAT= builtin history 1); export CURRENT_CMDLINE; nodestorage -i'
alias nodestorage='CURRENT_CMDLINE=$(HISTTIMEFORMAT= builtin history 1); export CURRENT_CMDLINE; nodestorage'
alias nodeshell='CURRENT_CMDLINE=$(HISTTIMEFORMAT= builtin history 1); export CURRENT_CMDLINE; nodeshell'
alias nodelicense='CURRENT_CMDLINE=$(HISTTIMEFORMAT= builtin history 1); export CURRENT_CMDLINE; nodelicense'
# Do not continue for non-bash shells, the rest of this sets up bash completion functions
@@ -82,17 +81,6 @@ function _confluent_generic_completion()
return;
fi
}
function _confluent_generic_ng_completion()
{
_confluent_get_args
if [ $NUMARGS -ge 3 ] && [ ! -z "$GENNED" ]; then
COMPREPLY=($(compgen -W "$GENNED" -- ${COMP_WORDS[COMP_CWORD]}))
fi
if [ $NUMARGS -lt 3 ]; then
_confluent_ng_completion
return;
fi
}
_confluent_nodeidentify_completion()
{
COMP_CANDIDATES=("on,off,blink -h")
@@ -102,7 +90,7 @@ _confluent_nodeidentify_completion()
_confluent_nodesetboot_completion()
{
COMP_CANDIDATES=("default,cd,network,setup,hd,floppy,usb -h -b -p")
COMP_CANDIDATES=("default,cd,network,setup,hd -h -b -p")
_confluent_generic_completion
}
@@ -149,71 +137,6 @@ _confluent_nodefirmware_completion()
fi
}
_confluent_osimage_completion()
{
_confluent_get_args
if [ $NUMARGS == 2 ]; then
COMPREPLY=($(compgen -W "initialize import importcheck updateboot rebase" -- ${COMP_WORDS[COMP_CWORD]}))
return
elif [ ${CMPARGS[1]} == 'initialize' ]; then
COMPREPLY=($(compgen -W "-h -u -s -t -i" -- ${COMP_WORDS[COMP_CWORD]}))
elif [ ${CMPARGS[1]} == 'import' ] || [ ${CMPARGS[1]} == 'importcheck' ]; then
compopt -o default
COMPREPLY=()
return
elif [ ${CMPARGS[1]} == 'updateboot' -o ${CMPARGS[1]} == 'rebase' ]; then
COMPREPLY=($(compgen -W "-n $(confetty show /deployment/profiles|sed -e 's/\///')" -- "${COMP_WORDS[COMP_CWORD]}"))
return
fi
}
_confluent_imgutil_completion()
{
_confluent_get_args
if [ $NUMARGS == 2 ]; then
COMPREPLY=($(compgen -W "build exec unpack pack capture" -- ${COMP_WORDS[COMP_CWORD]}))
return
elif [ ${CMPARGS[1]} == 'build' ]; then
COMPREPLY=($(compgen -W "-s $(ls /var/lib/confluent/distributions)" -- ${COMP_WORDS[COMP_CWORD]}))
return
elif [ ${CMPARGS[1]} == 'unpack' ]; then
if [ $NUMARGS == 3 ]; then
COMPREPLY=($(compgen -W "-s $(ls /var/lib/confluent/public/os)" -- ${COMP_WORDS[COMP_CWORD]}))
return
fi
compopt -o dirnames
return
elif [ ${CMPARGS[1]} == 'pack' ]; then
if [ $NUMARGS == 3 ]; then
compopt -o dirnames
fi
COMPREPLY=()
return
elif [ ${CMPARGS[1]} == 'capture' ]; then
if [ $NUMARGS == 3 ]; then
_confluent_nn_completion
return
fi
return
elif [ ${CMPARGS[1]} == 'exec' ]; then
compopt -o dirnames
COMPREPLY=($(compgen -W "-v" -- ${COMP_WORDS[COMP_CWORD]}))
return
fi
}
_confluent_nodedeploy_completion()
{
_confluent_get_args
if [ $NUMARGS -gt 2 ]; then
COMPREPLY=($(compgen -W "-n $(confetty show /deployment/profiles|sed -e 's/\///')" -- "${COMP_WORDS[COMP_CWORD]}"))
return
fi
if [ $NUMARGS -lt 3 ]; then
_confluent_nr_completion
return;
fi
}
_confluent_nodeshell_completion()
{
_confluent_get_args
@@ -279,23 +202,13 @@ _confluent_nodeattrib_completion()
_confluent_generic_completion
}
_confluent_define_completion()
{
COMP_CANDIDATES=$(nodegroupattrib 'everything' all | awk '{print $2}'|sed -e 's/://')
_confluent_get_args
if [ $NUMARGS -ge 3 ] && [ ! -z "$GENNED" ]; then
COMPREPLY=($(compgen -W "$GENNED" -- ${COMP_WORDS[COMP_CWORD]}))
fi
}
_confluent_nodegroupattrib_completion()
{
COMP_CANDIDATES=$(nodegroupattrib 'everything' all | awk '{print $2}'|sed -e 's/://')
_confluent_generic_ng_completion
}
_confluent_nn_completion()
{
_confluent_get_args
if [ $NUMARGS -gt 2 ]; then
return;
fi
INPUT=${COMP_WORDS[COMP_CWORD]}
INPUT=${INPUT##*,-}
INPUT=${INPUT##*,}
@@ -313,8 +226,6 @@ _confluent_nr_completion()
CMPARGS=($COMP_LINE)
_confluent_get_args
if [ $NUMARGS -gt 2 ]; then
compopt -o default
COMPREPLY=()
return;
fi
INPUT=${COMP_WORDS[COMP_CWORD]}
@@ -347,30 +258,15 @@ _confluent_ng_completion()
COMPREPLY=($(compgen -W "$(confetty show /nodegroups|sed -e 's/\///' -e s/^/$PREFIX/)" -- "${COMP_WORDS[COMP_CWORD]}"))
}
_confluent_nodediscover_completion()
{
_confluent_get_args
if [ $NUMARGS == 2 ]; then
COMPREPLY=($(compgen -W "list assign rescan clear subscribe unsubscribe register" -- ${COMP_WORDS[COMP_CWORD]}))
return;
fi
}
complete -F _confluent_nodediscover_completion nodediscover
complete -F _confluent_nodeattrib_completion nodeattrib
complete -F _confluent_nodeattrib_completion nodegroupattrib
complete -F _confluent_nr_completion nodebmcreset
complete -F _confluent_nodesetboot_completion nodeboot
complete -F _confluent_nr_completion nodeconfig
complete -F _confluent_nn_completion nodeconsole
complete -F _confluent_define_completion nodedefine
complete -F _confluent_define_completion nodegroupdefine
complete -F _confluent_nr_completion nodeeventlog
complete -F _confluent_nodefirmware_completion nodefirmware
complete -F _confluent_nodedeploy_completion nodedeploy
complete -F _confluent_osimage_completion osdeploy
complete -F _confluent_imgutil_completion imgutil
complete -F _confluent_nodegroupattrib_completion nodegroupattrib
complete -F _confluent_ng_completion nodegroupattrib
complete -F _confluent_ng_completion nodegroupremove
complete -F _confluent_nr_completion nodehealth
complete -F _confluent_nodeidentify_completion nodeidentify
+11 -22
View File
@@ -3,7 +3,7 @@ collate(1) -- Organize text input by node
## SYNOPSIS
`<other command> | collate [-a] [-b] [-d] [-w] [-g] [-s] [-c] [-r] [-l lognametemplate]`
`<other command> | collate [-a] [-d] [-w] [-s] [-c] [-r] [-l lognametemplate]`
## DESCRIPTION
@@ -21,41 +21,30 @@ node and group names sorted alphanumerically.
when output differs along a telling group boundary. For example, if
output suggests a large number of nodes are unreachable, abbreviate
showing 'rack1' as being unreachable may make more obvious a possible cause.
* `-b BASE`, `--base=BASE`:
Use given node as reference for comparison when using
-d, instead of using the most common result
* `-d`, `--diff`:
Express all but the most common result group in terms of diff from
the most common result group
* `-l`, `--log`:
Save output per node to individual log files, replacing {node} in the name
with the nodename of each
* `-w`, `--watch`:
Update results dynamically as data becomes available, rather than
waiting for the command to fully complete.
* `-g`, `--groupcount`:
Show count of output groups rather than the actual
output
* `-s`, `--skipcommon`:
Suppress printing of the most common result text group. This is used to
focus on stray output against a well known and expected result.
* `-c`, `--count`:
Print a count of the number of nodes in an output group under the
noderange.
noderange.
* `-r`, `--reverse`:
Rather than starting with most common to least common, start with
the least common and print the most common last.
* `-l LOG`, `--log=LOG`:
Save output per node to individual log files, replacing {node} in the name
with the nodename of each
* `-h`, `--help`:
Show help message and exit
* `-r`, `--reverse`:
Rather than starting with most common to least common, start with
the least common and print the most common last.
## EXAMPLES
* Organizing power state of multiple nodes:
-4
View File
@@ -7,7 +7,6 @@ collective(1) -- Check and manage a confluent collective
`collective join <server> [-i TOKEN]`
`collective show`
`collective gencert`
`collective delete`
## DESCRIPTION
@@ -37,9 +36,6 @@ on circumstances.
* `-i`:
Provide the token as an argument rather than interactively.
* `-h`, `--help`:
Show help message and exit
## EXAMPLES
* Inviting a server called mgt2:
-15
View File
@@ -15,21 +15,6 @@ be mostly an aid for developing client software, with
day to day administration generally being easier with
the various function specific commands.
## OPTIONS
* `-s SERVER:PORT`, `--server=SERVER:PORT`:
Confluent instance to connect to
* `-c PATH`, `--control=PATH`:
Path to offer terminal control
* `-m MINTIME`, `--mintime=MINTIME`:
Minimum time to run or else pause for input (used to
keep a terminal from closing quickly on error)
* `-h`, `--help`:
Show help message and exit
## COMMANDS
The CLI may be navigated by shell commands and some other
@@ -1,28 +0,0 @@
confluent2hosts(8) -- Generate /etc/hosts entries for nodes
========================================================
## SYNOPSIS
`confluent2hosts -i <ip expression> -n <name expression> <noderange>`
`confluent2hosts -a <noderange>`
## DESCRIPTION
`confluent2hosts` can be used to help generate /etc/hosts entries for a
noderange. There are two general approaches.
It can be used ad-hoc, using -i and -n to specify the address and name portions respectively. This accepts the standard confluent expression syntax, allowing for things like 172.30.1.{n1} or {node}.{dns.domain} or {bmc}.
It can also read from the confluent db, using `-a`. In this mode, each net.<value>.<attribute> group is pulled together into hosts lines. ipv4_address and ipv6_address fields are associated with the corresponding hostname attributes. You can use `-f` to put the FQDN first.
## EXAMPLES
* Generate /etc/hosts entries ad-hoc using default name:
`# confluent2hosts -i 10.2.3.{n1} d9-d12`
* Generate /etc/hosts entries ad-hoc using alternative name:
`# confluent2hosts -i 10.2.3.{n1} -n "{node}-alt {node}-alt.{dns.domain}" d9-d12`
* Generate /etc/hosts entries using the confluent DB as a reference:
`# confluent2hosts -a d9-d12`
+4 -23
View File
@@ -3,7 +3,8 @@ confluentdbutil(8) -- Backup or restore confluent database
## SYNOPSIS
`confluentdbutil [options] [dump|restore] <path>`
`confluentdbutil [options] <dump|restore> <path>`
## DESCRIPTION
@@ -15,31 +16,11 @@ the json files (password protected, removed from the files, or unprotected).
## OPTIONS
* `-p PASSWORD`, `--password=PASSWORD`:
* `-p`, `--password`:
If specified, information such as usernames and passwords will be encrypted
using the given password.
* `-i`, `--interactivepassword`:
Prompt for password.
* `-r`, `--redact`:
Indicates to replace usernames and passwords with a dummy string rather
than included.
* `-u`, `--unprotected`:
The keys.json file will include the encryption keys without any protection.
* `-s`, `--skipkeys`:
This specifies to dump the encrypted data without
dumping the keys needed to decrypt it. This is
suitable for an automated incremental backup, where an
earlier password protected dump has a protected
keys.json file, and only the protected data is needed.
keys do not change and as such they do not require
incremental backup.
* `-y`, `--yaml
Use YAML instead of JSON as file format
* `-h`, `--help`:
Show help message and exit
The keys.json file will include the encryption keys without any protection.
-30
View File
@@ -1,30 +0,0 @@
nodeapply(8) -- Execute command on many nodes in a noderange through ssh
=========================================================================
## SYNOPSIS
`nodeapply [options] <noderange>`
## DESCRIPTION
Provides shortcut access to a number of common operations against deployed
nodes. These operations include refreshing ssh certificates and configuration,
rerunning syncflies, and executing specified postscripts.
## OPTIONS
* `-k`, `--security`
Refresh SSH configuration (hosts.equiv and node SSH certificates)
* `-F`, `--sync`
Rerun syncfiles from deployed profile
* `-P SCRIPTS`, `--scripts=SCRIPTS`
Re-run specified scripts, with full path under scripts specified, e.g. post.d/scriptname,firstboot.d/otherscriptname
* `-c COUNT`, `-f COUNT`, `--count=COUNT`
Specify the maximum number of instances to run concurrently
* `-m MAXNODES`, `--maxnodes=MAXNODES`
Specify a maximum number of nodes to run remote ssh command to, prompting
if over the threshold
+9 -54
View File
@@ -7,8 +7,7 @@ nodeattrib(8) -- List or change confluent nodes attributes
`nodeattrib <noderange> [<nodeattribute1=value1> <nodeattribute2=value2> ...]`
`nodeattrib -c <noderange> <nodeattribute1> <nodeattribute2> ...`
`nodeattrib -e <noderange> <nodeattribute1> <nodeattribute2> ...`
`nodeattrib -p <noderange> <nodeattribute1> <nodeattribute2> ...`
`nodeattrib <noderange> -s <attributes.batch> ...`
`nodeattrib -p <noderange> <nodeattribute1> <nodeattribute2> ...`
## DESCRIPTION
@@ -35,43 +34,18 @@ For the `groups` attribute, it is possible to add a group by doing
`groups,=<newgroup>` and to remove by doing `groups^=<oldgroup>`
Note that `nodeattrib <group>` will likely not provide the expected behavior.
See nodegroupattrib(8) command on how to manage attributes on a group level. Running
nodeattrib on a group will simply set node-specific attributes on each individual
member of the group.
Additionally, if wanting to change from a node level attribute value to inherit from a
member group, it is required to use '-c' to clear the attribute. Conversely, assigning
to a blank value will allow masking a group defined attribute with an empty value.
See nodegroupattrib(8) command on how to manage attributes on a group level.
## OPTIONS
* `-b`, `--blame`:
Annotate inherited and expression based attributes to show their base value.
* `-c`, `--clear`:
Clear specified nodeattributes.
Clear specified nodeattributes
* `-e`, `--environment`:
Set specified attributes based on exported environment variable of matching name.
Environment variable names may be lower case or all upper case.
Replace . with _ as needed (e.g. info.note may be specified as either $info_note or $INFO_NOTE)
Set specified attributes based on exported environment variable of matching name. Environment variable names may be lower case or all upper case. Replace . with _ as needed (e.g. info.note may be specified as either $info_note or $INFO_NOTE
* `-p`, `--prompt`:
Request interactive prompting to provide values rather than the command line
or environment variables.
* `-s`, `--set`:
Set attributes using a batch file rather than the command line. The attributes in the batch file
can be specified as one line of key=value pairs simmilar to command line or each attribute can
be in its own line. Lines that start with # sign will be read as a comment. See EXAMPLES for batch
file syntax.
* `-m MAXNODES`, `--maxnodes=MAXNODES`:
Prompt if trying to set attributes on more than
specified number of nodes.
* `-h`, `--help`:
Show help message and exit
Request interactive prompting to provide values rather than the command line or environment variables.
## EXAMPLES
* Listing matching nodes of a simple noderange:
@@ -90,12 +64,12 @@ to a blank value will allow masking a group defined attribute with an empty valu
`# nodeattrib n1,n2 hardwaremanagement --blame`
`n1: hardwaremanagement.manager: 172.30.3.1`
`n1: hardwaremanagement.method: ipmi (inherited from group everything)`
`n1: net.switch: r8e1`
`n1: net.switchport: 14`
`n1: hardwaremanagement.switch: r8e1`
`n1: hardwaremanagement.switchport: 14`
`n2: hardwaremanagement.manager: 172.30.3.2`
`n2: hardwaremanagement.method: ipmi (inherited from group everything)`
`n2: net.switch: r8e1`
`n2: net.switchport: 2`
`n2: hardwaremanagement.switch: r8e1`
`n2: hardwaremanagement.switchport: 2`
* Listing matching nodes of a simple noderange that are set:
`# nodeattrib n1-n2 current`
@@ -123,25 +97,6 @@ to a blank value will allow masking a group defined attribute with an empty valu
`d1: net.pxe.switch: pxeswitch1`
`d1: net.switch:`
* Setting attributes using a batch file with syntax similar to command line:
`# cat nodeattributes.batch`
`# power`
`power.psu1.outlet=3 power.psu1.pdu=pdu2`
`# nodeattrib n41 -s nodeattributes.batch`
`n41: 3`
`n41: pdu2`
* Setting attributes using a batch file with syntax where each attribute is in its own line:
`# cat nodeattributes.batch`
`# management`
`custom.mgt.switch=switch_main`
`custom.mgt.switch.port=swp4`
`# nodeattrib n41 -s nodeattributes.batch`
`n41: switch_main`
`n41: swp4`
## SEE ALSO
nodegroupattrib(8), nodeattribexpressions(5)
@@ -1,28 +0,0 @@
nodebmcpassword(8) -- Change management controller password for a specified user
=========================================================
## SYNOPSIS
`nodebmcpassword <noderange> <username> <new_password>`
## DESCRIPTION
`nodebmcpassword` allows you to change the management controller password for a user on a specified noderange
## OPTIONS
* `-m MAXNODES`, `--maxnodes=MAXNODES`:
Number of nodes to affect before prompting for
confirmation
* `-h`, `--help`:
Show help message and exit
## EXAMPLES:
* Reset the management controller for nodes n1 through n4:
`# nodebmcreset n1-n4`
`n1: Password Change Successful`
`n2: Password Change Successful`
`n3: Password Change Successful`
`n4: Password Change Successful`
@@ -9,15 +9,6 @@ nodebmcreset(8) -- Reset management controller
`nodebmcreset` allows you to reset the management controller of the specified noderange
## OPTIONS
* `-m MAXNODES`, `--maxnodes=MAXNODES`:
Number of nodes to affect before prompting for
confirmation
* `-h`, `--help`:
Show help message and exit
## EXAMPLES:
* Reset the management controller for nodes n1 through n4:
+2 -8
View File
@@ -3,7 +3,8 @@ nodeboot(8) -- Reboot a confluent node to a specific device
## SYNOPSIS
`nodeboot [options] <noderange> [default|cd|network|setup|hd]`
`nodeboot <noderange>`
`nodeboot [options] <noderange>` [default|cd|network|setup|hd]
## DESCRIPTION
@@ -24,13 +25,6 @@ device without inducing a reboot, see the `nodesetboot` command.
* `-p`, `--persist`:
For a system that supports it, mark the boot override to persist rather than
be a one time change. Many systems do not support this functionality.
* `-m MAXNODES`, `--maxnodes=MAXNODES`:
Specify a maximum number of nodes to boot, prompting
if over the threshold
* `-h`, `--help`:
Show help message and exit
* `default`:
Request a normal default boot with no particular device override
+9 -24
View File
@@ -3,7 +3,8 @@ nodeconfig(8) -- Show or change node configuration
## SYNOPSIS
`nodeconfig [options] <noderange> [setting|setting=value]`
`nodeconfig <noderange> [options] [<configuration>..]`
`nodeconfig <noderange> [options] [<configuration=value>..]`
## DESCRIPTION
@@ -13,12 +14,6 @@ running configuration on the node firmware. Calling without '=' will show the
current value, and '=' will change the value. Network information can be
given as a node expression, as documented in the man page for nodeattribexpressions(5).
Note that when using nodeconfig to submit changes, it will exit when the change
is accepted, but the endpoint may not have fully processed it. Doing a show
immediately after doing a set may reflect older information. Also, if changing
BIOS/UEFI settings, the change may appear in output, but generally won't
actually be in effect until a reboot.
## OPTIONS
* `-c`, `--comparedefault`:
@@ -26,13 +21,6 @@ actually be in effect until a reboot.
no configuration values are specified, it will show only those that differ.
If combined with `-x`, will show all differing values except those indicated
by `-x`
* `-b settings.batch`, `--batch=settings.batch`:
Provide arguments as lines of a file, rather than the command line.
* `-d`, `--detail`:
Provide detailed data as available. This can include help text and valid
values for a setting.
* `-e`, `--extra`:
Read settings that are generally not needed, but may be slow to retrieve.
@@ -43,21 +31,18 @@ actually be in effect until a reboot.
Rather than listing only the specified configuration parameters, list all
attributes except for the specified ones
* `-a`, `--advanced`:
Include advanced settings, which are normally not intended to be used
without direction from the relevant server vendor.
* `-d`, `--detail`:
Provide detailed data as available. This can include help text and valid
values for a setting.
* `-r COMPONENT`, `--restoredefault=COMPONENT`:
* `-r`, `--restoredefault`:
Request that the specified component of the targeted nodes will have its
configuration reset to default. Currently the only component implemented
is uefi.
* `-m MAXNODES`, `--maxnodes=MAXNODES`:
Specify a maximum number of nodes to configure, prompting if over
the threshold
* `-h`, `--help`:
Show help message and exit
* `-b`, '--batch':
Provide arguments as lines of a file, rather than the command line.
## EXAMPLES
* Showing the current IP configuration of noderange BMC/IMM/XCC:
+1 -45
View File
@@ -2,7 +2,7 @@ nodeconsole(8) -- Open a console to a confluent node
=====================================================
## SYNOPSIS
`nodeconsole [options] <noderange> [kill][-- [passthroughoptions]]`
`nodeconsole [options] <noderange>`
## DESCRIPTION
@@ -16,45 +16,12 @@ will initiate an automatic retry interval that is randomized between 2 and 4 min
The reopen escape sequence below requests an immediate retry, as does connecting
a new session.
When a windowed console is open the `nodeconsole <noderange> kill` command will kill the
console process which will result in the console window closing.
## OPTIONS
* `-t`, `--tile`:
Use tmux to arrange consoles of the given noderange into a tiled layout on
the terminal screen
* `-l`, `--log`:
Perform a log reply on the current, local log in /var/log/confluent/consoles.
If in collective mode, this only makes sense to use on the current collective
manager at this time.
* `-T`, `--Timestamp`:
Dump the log with Timpstamps on the current, local log in /var/log/confluent/consoles.
If in collective mode, this only makes sense to use on the current collective
manager at this time.
* `-w`, `--windowed`:
Open terminal windows for each node. The
environment variable **NODECONSOLE_WINDOWED_COMMAND**
should be set, which should be a text string corresponding
to a command that can be used to open a windowed console,
omitting the `nodeconsole <noderange>` part of the
command, for example, to open a set of consoles for a
range of nodes in separate xterm windows, set
**NODECONSOLE_WINDOWED_COMMAND** to `xterm -e`. To open a
set of consoles for a range of nodes in separate
GNOME Terminal windows with a size of 100 columns and
31 rows, set **NODECONSOLE_WINDOWED_COMMAND**
to `gnome-terminal --geometry 100x31 --` or in a WSL
environment, to open a set of consoles for a range of
nodes in separate Windows Terminal windows, with the
title set for each node, set **NODECONSOLE_WINDOWED_COMMAND**
to `wt.exe wsl.exe -d AlmaLinux-8 --shell-type login. If the
NODECONSOLE_WINDOWED_COMMAND environment variable isn't set,
xterm will be used bydefault.
## ESCAPE SEQUENCE COMMANDS
While connected to a console, a number of commands may be performed through escape
@@ -94,14 +61,3 @@ keystroke will be interpreted as a command. The following commands are availabl
Get a list of supported commands
* `<ent>`:
Hit enter to skip entering a command at the escape prompt.
## PASSTHROUGH OPTIONS
While opening a windowed console with xterm or any other console of choice. The
nodeconsole command gives capality to specify passthrough options targeted at
the console. All options after the -- will be parsed the console program. For
example, opening a windowed console using xterm with a black background.
`nodeconconsole -w n1 -- -bg black`
-46
View File
@@ -1,46 +0,0 @@
nodedeploy(8) -- Request preparation and/or initiating a node deployment
=========================================================================
## SYNOPSIS
`nodedeploy [-h] [-c] [-n] [-p] [-m MAXNODES] <noderange> [profile]`
## DESCRIPTION
**nodedeploy** configures attributes and interacts with BMC devices as appropriate
to change the operating system on the noderange to that indicated by `profile`. Tab completion
is supported under bash to help list and select a profile name. `-n` indicates that it
will be a network based deployment (either HTTP or PXE) and `-p` suppresses any activity
that requires the BMC (e.g. setting the next boot device and rebooting the system). Currently
**nodedeploy** only supports `-n` style deployment flows. Without options it shows the current
deployment status.
## OPTIONS
* `-c`, `--clear`:
Remove any pending deployment action
* `-n`, `--network`:
Prepare for either an HTTP or PXE based deployment, setting boot device to network and rebooting unless `-p` is specified.
* `-p`, `--prepareonly`:
Prepare the network services for deployment, but do not interact with BMCs. This is intended for scenarios where
the boot device control and server restart will be handled outside of confluent.
* `-m MAXNODES`, `--maxnodes=MAXNODES`:
Specifiy a maximum nodes to be deployed.
* `-h`, `--help`:
Show help message and exit
## EXAMPLES
* Begin the instalalation of a profile of CentOS 8.2:
`# nodedeploy d4 -n centos-8.2-x86_64-default`
`d4: network`
`d4: reset`
* Check current deployment state of nodes:
`# nodedeploy d4`
`d4: pending: centos-8.2-x86_64-default (node authentication armed)`
+3 -12
View File
@@ -5,12 +5,9 @@ nodediscover(8) -- List or manage confluent node discovery
`nodediscover rescan`
`nodediscover [options] list`
`nodediscover [options] assign`
`nodediscover [options] rescan`
`nodediscover [options] clear`
`nodediscover [options] subscribe [switch]`
`nodediscover [options] unsubscribe [switch]`
`nodediscover [options] register [Addresses, e.g. 192.168.1.2 or 192.168.1.0/24 or 192.168.1.1-192.168.1.28]`
`nodediscover [options] assign`
`nodediscover [options] rescan`
`nodediscover [options] clear`
## DESCRIPTION
@@ -38,12 +35,6 @@ scan is the best approach to catch something that appears to be missing.
**nodedsicover clear** requests the server forget about a selection of
detected device. It takes the same arguments as **nodediscover list**.
**nodediscover subscribe** and **unsubscribe** instructs confluent to subscribe to or
unsubscribe from the designated switch running affluent with system discovery support.
**nodediscover register** instructs confluent to perform a remote probe of an address, subnet,
or range of IP addresses.
## CSV FORMAT
The CSV format used by nodediscover consists of one header describing the
+2 -19
View File
@@ -3,31 +3,14 @@ nodeeventlog(8) -- Pull eventlog from confluent nodes
## SYNOPSIS
`nodeeventlog [options] <noderange> [clear]`
`nodeeventlog <noderange>`
`nodeeventlog <noderange> [clear]`
## DESCRIPTION
`nodeeventlog` pulls and optionally clears the event log from the requested
noderange.
## OPTIONS
* `-m MAXNODES`, `--maxnodes=MAXNODES`:
Specify a maximum number of nodes to clear if clearing log, prompting if
over the threshold
* `-l LINES`, `--lines=LINES`:
return the last <n> entries for each node in the eventlog.
* `-t TIMEFRAME`, `--timeframe=TIMEFRAME`:
return entries within a specified timeframe for each node's event log.
This will return entries from the last <n> hours or days. 1h would be
entries from with the last one hour.
* `-h`, `--help`:
Show help message and exit
## EXAMPLES
* Pull the event log from n2 and n3:
`# nodeeventlog n2,n3`
+3 -16
View File
@@ -3,7 +3,9 @@ nodefirmware(8) -- Report firmware information on confluent nodes
## SYNOPSIS
`nodefirmware <noderange> [list][updatestatus][update [--backup <file>]]|[<components>]`
`nodefirmware <noderange>`
`nodefirmware <noderange> list|<components>|core`
`nodefirmware <noderange> update [--backup] <filename>`
## DESCRIPTION
@@ -17,9 +19,6 @@ not be relevant to redfish. Additionally, the Lenovo XCC makes certain
information available over IPMI that is not otherwise available (for example
the FPGA version where applicable).
The updatestatus argument will describe the state of firmware updates on the
nodes.
In the update form, it accepts a single file and attempts to update it using
the out of band facilities. Firmware updates can end in one of three states:
@@ -27,18 +26,6 @@ the out of band facilities. Firmware updates can end in one of three states:
* `pending`: The firmware update process has completed, but the firmware will not be active until the relevant component next resets. Generally speaking, for UEFI update the system will need a reboot, and for BMC updates, the `nodebmcreset` command will begin the process to activate the firmware.
* `complete`: The firmware update process has completed and activation has proceeded. Note that while the activation process has commenced, the component may still be in the process of rebooting when nodefirmware exits.
## OPTIONS
* `-b`, `--backup`:
Target a backup bank rather than primary
* `-m MAXNODES`, `--maxnodes=MAXNODES`:
When updating, prompt if more than the specified number of servers will
be affected
* `-h`, `--help`:
Show help message and exit
## EXAMPLES
* Pull firmware from a node:
@@ -25,20 +25,8 @@ the attributes are set on the node versus a group to which a node belongs.
## OPTIONS
* `-b`, `--blame`:
Show information about how attributes inherited
* `-e`, `--environment`:
Set attributes, but from environment variable of same name
* `-c`, `--clear`:
Clear specified nodeattributes.
* `-p`, `--prompt`:
Prompt for attribute values interactively
* `-h`, `--help`:
Show help message and exit
## EXAMPLES
@@ -1,42 +0,0 @@
nodel2traceroute(8) -- returns the layer 2 route through an Ethernet network managed by confluent given 2 end points.
==============================
## SYNOPSIS
`nodel2traceroute [options] <start_node> <end_noderange>`
## DESCRIPTION
**nodel2traceroute** is a command that returns the layer 2 route for the configered interfaces in nodeattrib.
It can also be used with the -i and -e options to check against specific interfaces on the endpoints. If the
--interface or --eface option are not used then the command will check for routes against all the defined
interfaces in nodeattrib (net.*.switch) for the nodes.
## PREREQUISITES
**nodel2traceroute** the net.<interface>.switch attributes have to be set on the end points if endpoint is not a switch
## OPTIONS
* ` -e` EFACE, --eface=INTERFACE
interface to check against for the second end point or end points if using checking against multiple nodes
* ` -i` INTERFACE, --interface=INTERFACE
interface to check against for the first end point
* ` -c` CUMULUS, --cumulus=CUMULUS
return layer 2 route through cumulus switches only
* `-h`, `--help`:
Show help message and exit
## EXAMPLES
* Checking route between two nodes:
`# nodel2traceroute n244 n1851`
`n244 to n1851: ['switch114']`
* Checking route from one node to multiple nodes:
`# nodel2traceroute n244 n1833,n1851`
`n244 to n1833: ['switch114', 'switch7', 'switch32', 'switch253', 'switch85', 'switch72', 'switch21', 'switch2', 'switch96', 'switch103', 'switch115']
n244 to n1851: ['switch114']`
+2 -9
View File
@@ -3,22 +3,15 @@ nodelicense(8) -- Manage license keys on BMC
## SYNOPSIS
`nodelicense <noderange> [list][install <license_filename>|save <directory>|delete <license_feature_name>]`
`nodelicense <noderange> [list|install <filename>|delete <license>|save <directory>]`
## DESCRIPTION
`nodelicense` manages license keys on supported BMCs. Without an argument, the command
lists currently installed license. Using `delete` will remove the specified license name
from the BMC. The `save` subcommand will take the passed directory (which may be in the form
from th eBMC. The `save` subcommand will take the passed directory (which may be in the form
of /path/to/{node}/ to have the node name substituted for each node) and back up installed licenses
to that directory. The `install` command will take the specified filename and install. The filename
argument may be of the form xcc_fod_0034_7X21{id.serial}.key to have the serial number substituted
to allow unique licenses to be specified in a single command.
## OPTIONS
* `-m MAXNODES`, `--maxnodes=MAXNODES`:
Specify a maximum number of nodes to delete licenses from, prompting if over the threshold
* `-h`, `--help`:
Show help message and exit
+2 -3
View File
@@ -4,7 +4,7 @@ nodelist(8) -- List confluent nodes and their attributes
## SYNOPSIS
`nodelist <noderange>`
`nodelist <noderange> [-b] [-d] {string} <nodeattribute>...`
`nodelist <noderange> [-b] <nodeattribute>...`
## DESCRIPTION
@@ -24,8 +24,7 @@ all attributes that begin with `net.` and end with `switch`.
* `-b`, `--blame`:
Annotate inherited and expression based attributes to show their base value.
* `-d`, `--delim`:
Choose a delimiter to separat the values. Default - ENTER.
## EXAMPLES
* Listing matching nodes of a simple noderange:
`# nodelist n1-n4`
+2 -2
View File
@@ -34,8 +34,8 @@ the host platform attempts.
## OPTIONS
* `-h`, `--help`:
Show help message and exit
* `-c`:
Specify the maximum number of instances to run concurrently.
## EXAMPLES
* Listing currently mounted media:
-47
View File
@@ -1,47 +0,0 @@
nodeping(8) -- Pings a node or a noderange.
==============================
## SYNOPSIS
`nodeping [options] noderange`
## DESCRIPTION
**nodeping** is a command that pings the default NIC on a node.
It can also be used with the `-s` flag to change the ping location to something that is 'non primary'
## OPTIONS
* ` -f` COUNT, `-c` COUNT, --count=COUNT
Number of commands to run at a time
* `-h`, `--help`:
Show help message and exit
* `-s` SUBSTITUTENAME, --substitutename=SUBSTITUTENAME
Use a different name other than the nodename for ping. This may be a
expression, such as {bmc} or, if no { character is present, it is treated as a suffix. -s -eth1 would make n1 become n1-eth1, for example.
## EXAMPLES
* Pinging a node :
`# nodeping <node>`
`node : ping`
* Pinging a group:
`# nodeping <groupname>`
`Node1 : ping
Node2 : ping
Node3 : ping`
* Pinging BMC on a node:
`# nodeping -s {bmc} <noderange>`
` Node-bmc : ping`
* Pinging by specifying a suffix:
`# nodeping d1-d4 -s -eth1`
`d2-eth1: no_ping`
`d1-eth1: no_ping`
`d3-eth1: no_ping`
`d4-eth1: no_ping`
* Fail to ping node:
`# nodeping <node>`
`node : no_ping`
+3 -12
View File
@@ -3,7 +3,8 @@ nodepower(8) -- Check or change power state of confluent nodes
## SYNOPSIS
`nodepower [options] <noderange> ([status|on|off|shutdown|boot|reset])`
`nodepower <noderange>`
`nodepower <noderange> [-p] [on|off|boot|shutdown|reset|status]`
## DESCRIPTION
@@ -24,21 +25,11 @@ respond.
* `reset`: Request immediate reset of nodes of the noderange. Nodes that are
off will not react to this request.
* `status`: Behave identically to having no argument passed at all.
* `pdu_status`: Query state of associated PDU outlets, if configured.
* `pdu_on`: Energize all PDU outlets associated with the noderange.
* `pdu_off`: De-energize all PDU outlets associated with the noderange.
## OPTIONS
* `-p`, `--showprevious`:
* `-p`, '--showprevious':
Show previous power state for all directives that may change power state.
* `-m MAXNODES`, `--maxnodes=MAXNODES`:
Specify a maximum number of nodes to change power state, prompting if
over the threshold
* `-h`, `--help`:
Show help message and exit
## EXAMPLES
* Get power state of nodes n1 through n4:
+1 -8
View File
@@ -9,14 +9,6 @@ noderemove(8) -- Remove nodes from the confluent management service
`noderemove` simply removes the given noderange from the confluent database.
## OPTIONS
* `-m MAXNODES`, `--maxnodes=MAXNODES`:
Specify a maximum number of nodes to delete, prompting if over the
threshold
* `-h`, `--help`:
Show help message and exit
## EXAMPLES
@@ -30,3 +22,4 @@ noderemove(8) -- Remove nodes from the confluent management service
`r2u3: deleted`
`r2u2: deleted`
`r2u1: deleted`
+1 -8
View File
@@ -11,16 +11,9 @@ nodereseat(8) -- Request a reseat of a node
node's slot. This should be equivalent to removing the system entirely from
the chassis and putting it back in, but without actually having to do so.
## OPTIONS
* `-m MAXNODES`, `--maxnodes=MAXNODES`:
Specify a maximum number of nodes to reseat, prompting if over the threshold
* `-h`, `--help`:
Show help message and exit
## EXAMPLES
* Reseating the node `s1`:
`# nodereseat s1`
`s1: Reseat successful`
+5 -12
View File
@@ -12,17 +12,10 @@ noderange. This will present progress as percentage for all nodes.
## OPTIONS
* `-f COUNT`, `-c COUNT`, `--count=COUNT`:
* `-m`:
Specify maximum number of nodes for noderange max.
* `-c`:
Specify how many rsync executions to do concurrently. If noderange
exceeds the count, then excess nodes will wait until one of the
active count completes.
* `-s`, `--substitutename`:
'Use a different name other than the nodename for rsync'
* `-m MAXNODES`, `--maxnodes=MAXNODES`:
Specify a maximum number of nodes to run rsync to, prompting if over the
threshold
* `-h`, `--help`:
Show help message and exit
active count completes.
+1 -16
View File
@@ -3,7 +3,7 @@ noderun(8) -- Run arbitrary commands per node in a noderange
## SYNOPSIS
`noderun [options] <noderange> <command expression>`
`noderun <noderange> <command string>`
## DESCRIPTION
@@ -13,21 +13,6 @@ nodeattribexpressions(5) are expanded prior to execution of the command. For
noderun, the commands are locally executed. To execute commands on the nodes
themselves, see nodeshell(8).
## OPTIONS
* `-f COUNT`, `-c COUNT`, `--count=COUNT`:
Number of commands to run at a time
* `-n`, `--nonodeprefix`:
Do not prefix output with node names
* `-m MAXNODES`, `--maxnodes=MAXNODES`:
Specify a maximum number of nodes to run the command with, prompting if over
the threshold
* `-h`, `--help`:
Show help message and exit
## EXAMPLES
* Run ping against nodes n1 through n4:
@@ -29,10 +29,6 @@ interval of 1 second is used.
Perform the specified number of readings, waiting `-i` indicated interval or 1 second if not
otherwise indicated.
* `-s`, `--skipnumberless`:
Return only sensors that provide numeric data (e.g. temperature, fanspeed),
ignoring sensors that do not provide numeric data.
## EXAMPLES
* Retrieving all temperature related sensors from one system
`# nodesensors n1 temperature`
+3 -19
View File
@@ -2,8 +2,9 @@ nodesetboot(8) -- Check or set next boot device for noderange
====================================================
## SYNOPSIS
`nodesetboot [options] <noderange> [default|cd|network|setup|hd|usb|floppy]`
`nodesetboot <noderange>`
`nodesetboot [options] <noderange> [default|cd|network|setup|hd]`
## DESCRIPTION
@@ -20,8 +21,6 @@ to use `nodesetboot <noderange> setup` and then initiate a reboot from within
the operating system with ssh or similar rather than using the remote hardware
control.
Running the command with no target queries the current setting.
## OPTIONS
* `-b`, `--bios`:
@@ -34,13 +33,6 @@ Running the command with no target queries the current setting.
* `-u`, `--uefi`:
This flag does nothing, it is for command compatibility with xCAT's rsetboot
* `-m MAXNODES`, `--maxnodes=MAXNODES`:
Specify a maximum number of nodes to modify next boot device, prompting if
over the threshold
* `-h`, `--help`:
Show help message and exit
* `default`:
Request a normal default boot with no particular device override
@@ -50,14 +42,6 @@ Running the command with no target queries the current setting.
remote media mounted as CD/DVD, and detachable hard disks drives such as usb
key devices.
* `floppy`:
Request boot from floppy. Generally speaking firmware uses this to mean a USB
flash drive or similar (whether virtual or physical).
* `usb`:
Request boot from usb. Generally speaking firmware uses this to mean a USB
flash drive or similar (whether virtual or physical).
* `network`:
Request boot to network
+3 -23
View File
@@ -3,7 +3,7 @@ nodeshell(8) -- Execute command on many nodes in a noderange through ssh
## SYNOPSIS
`nodeshell [options] <noderange> <command to execute on each node>`
`nodeshell <noderange> [options] <command to execute on each node>`
## DESCRIPTION
@@ -13,29 +13,9 @@ nodeattribexpressions(5). `nodeshell` provides stdout as stdout and stderr
as stderr, unlike psh which combines all stdout and stderr into stdout.
## OPTIONS
* `-c COUNT`, `-f COUNT`, `--count=COUNT`
Specify the maximum number of instances to run concurrently
* `-l LOGINNAME`, `--loginname=LOGINNAME`
Username to use when connecting, defaults to current user.
* `-m MAXNODES`, `--maxnodes=MAXNODES`
Specify a maximum number of nodes to run remote ssh command to, prompting
if over the threshold
* `-n`, `--nonodeprefix`
Do not prefix output with node names
* `-p PORT`, `--port=PORT`
Specify a custom port for ssh
* `-s SUBSTITUTION`, `--substitutename=SUBSTITITUTION`
Specify a substitution name instead of the nodename. If no {} are in the substitution,
it is considered to be an append. For example, '-s -ib' would produce 'node1-ib' from 'node1'.
Full expression syntax is supported, in which case the substitution is considered to be the entire
new name. {node}-ib would be equivalent to -ib. For example, nodeshell -s {bmc} node1
would ssh to the BMC instead of the node.
* `-c`:
Specify the maximum number of instances to run concurrently.
## EXAMPLES
+2 -14
View File
@@ -3,15 +3,12 @@ nodestorage(8) -- Examine/Modify storage configuration of a node
## SYNOPSIS
`nodestorage <noderange> [show|create|delete|diskset] [hotspare|jbod|unconfigured] [options]`
`nodestorage <noderange> [show|create|delete] [options]`
## DESCRIPTION
`nodestorage` provides access to the remote storage configuration of
the noderange. The `show` subcommand will show current storage configuration,
`create` can be used to create new arrays or volumes, `delete` can be used to
remove volumes and arrays, and `diskset` can modify the usage of disks
indicated by `-d` to either be `unconfigured`, `jbod`, or `hotspare`.
the noderange.
## OPTIONS
* `-r` **RAIDLEVEL**, `--raidlevel`=**RAIDLEVEL**:
@@ -32,15 +29,6 @@ indicated by `-d` to either be `unconfigured`, `jbod`, or `hotspare`.
volumes, or selecting a volume for delete. Default
behavior is to use implementation provided default
* `-z` **STRIPSIZES**, `--stripsizes`=**STRIPSIZES**:
Comma separated list of stripsizes to use when creating volumes.
This value is in kilobytes. The default behavior is to allow the
storage controller to decide
* `-m` **MAXNODES**, `--maxnodes`=**MAXNODES**:
Specify a maximum number of nodes to configure storage on, prompting
if over the threshold
## EXAMPLES
* Deleting the volume `somedata`:
@@ -19,15 +19,6 @@ running in collective mode.
Note that due to vendor filename requirements, any filename may have vendor
specific suffixes added to any file produced.
## OPTIONS
* `-m MAXNODES`, `--maxnodes=MAXNODES`:
Specify a maximum number of nodes to download diagnostic data from, prompting
if over the threshold
* `-h`, `--help`:
Show help message and exit
## EXAMPLES
* Download support data from a single node to a specific filename
-19
View File
@@ -1,19 +0,0 @@
osdeploy(8) --- Configure general OS deployment facilities of confluent
==========================================================================
## SYNOPSIS
`osdeploy import <iso>`
`osdeploy updateboot <profile>`
`osdeploy rebase <profile>`
`osdeploy initialize [-h] [-g] [-u] [-s] [-k] [-t] [-p] [-i] [-l] [-a]`
## DESCRIPTION
**osdeploy** manages the facilities and os deployment content of a confluent server. The
`import` subcommand will generate profiles from an iso image. `updateboot` will take any
changes in the specified update that need to be pushed into boot configuration and/or images
and ensure those needed changes are performed. `initialize` provides assistance in setting
up the most commonly required facilities. `rebase` has confluent attempt to update profile
content that came from /opt/confluent in an rpm update, if profile supports it.
Run `osdeploy initialize -h` for more detail on the options offered by `osdeploy initialize`.
-49
View File
@@ -1,49 +0,0 @@
stats(8) -- Common basic statistics on typical numeric data in output
==============================
## SYNOPSIS
`<other command> | stats [-c N] [-d D] [-x|-g|-t|-o image.png] [-s N] [-v] [-b N]
## DESCRIPTION
The **stats** command helps analyze common numerical data such as performance numbers
or temperatures or any other numerical value.
By default it looks for the last numerical output on the first line to identify the numerical column
and analyze that number. This can be overriden by **-c COLUMN** to indicate a column. By default,
whitespace and commas are treated to delimit columns, and **-d DELIMITER** can be used to override.
By default it outputs basic statistics, but a histogram is available either text or through X11 output
or sixel or output to an image file depending on whether **-x**, **-g**, **-t*, or **-o image.png** is
used.
## OPTIONS
* `-c N`:
Select column number. Defaults to last column that appears numeric
* `-d D`:
Specify a custom column delimiter
* `-x`:
Output in Sixel format (supported by mlterm and PuTTY, among others)
* `-g`:
Try to open histogram as an X window
* `-t`:
Output histogram as bars rendered by =
* `-o image.png`:
Write graphical histogram to image.png.
* `-s N`:
Ignore specified number of lines as header content before processing numbers
* `-v`:
Treat value before : on each line as a label, and show which labels belong to which histogram buckets.
* `-b N`:
Specify a custom number of buckets for histogram. The default is 10.
+3 -9
View File
@@ -1,9 +1,7 @@
from setuptools import setup
import os
data_files = [('/etc/profile.d', ['confluent_env.sh', 'confluent_env.csh']),
('/opt/confluent/share/licenses/confluent_client/', ['LICENSE', 'COPYRIGHT'])
]
data_files = [('/etc/profile.d', ['confluent_env.sh', 'confluent_env.csh'])]
try:
scriptlist = ['bin/{0}'.format(d) for d in os.listdir('bin/')]
data_files.append(('/opt/confluent/share/man/man1', ['man/man1/' + x for x in os.listdir('man/man1')]))
@@ -15,14 +13,10 @@ except OSError:
setup(
name='confluent_client',
version='#VERSION#',
author='Lenovo',
author='Jarrod Johnson',
author_email='jjohnson2@lenovo.com',
url='http://github.com/lenovo/confluent/',
license='Apache-2.0',
description='Command line client and libraries for confluent management server',
long_description='Command line client and libraries for confluent management server',
url='http://xcat.sf.net/',
packages=['confluent'],
platforms=['Linux'],
scripts=scriptlist,
data_files=data_files,
)
-38
View File
@@ -1,38 +0,0 @@
VERSION=`git describe|cut -d- -f 1`
NUMCOMMITS=`git describe|cut -d- -f 2`
if [ "$NUMCOMMITS" != "$VERSION" ]; then
LASTNUM=$(echo $VERSION|rev|cut -d . -f 1|rev)
LASTNUM=$((LASTNUM+1))
FIRSTPART=$(echo $VERSION|rev|cut -d . -f 2- |rev)
VERSION=${FIRSTPART}.${LASTNUM}
VERSION=$VERSION~dev$NUMCOMMITS+`git describe|cut -d- -f 3`
fi
sed -e "s/#VERSION#/$VERSION/" confluent_osdeploy.spec.tmpl > confluent_osdeploy.spec
cd ..
cp ../LICENSE .
tar Jcvf confluent_osdeploy.tar.xz confluent_osdeploy
mv confluent_osdeploy.tar.xz ~/rpmbuild/SOURCES/
cd -
mkdir -p el9bin/opt/confluent/bin
mkdir -p el9bin/stateless-bin
podman run --privileged --rm -v $(pwd)/utils:/buildutils -i -t fedorabuild make -C /buildutils
cd utils
cp confluent_imginfo copernicus clortho autocons ../el9bin/opt/confluent/bin
cp start_root urlmount ../el9bin/stateless-bin/
make clean
cd ..
mkdir -p el8bin/opt/confluent/bin
mkdir -p el8bin/stateless-bin
podman run --privileged --rm -v $(pwd)/utils:/buildutils -i -t el7build make -C /buildutils
cd utils
cp confluent_imginfo copernicus clortho autocons ../el8bin/opt/confluent/bin
cp start_root urlmount ../el8bin/stateless-bin/
make clean
cd ..
tar Jcvf confluent_el9bin.tar.xz el9bin/
tar Jcvf confluent_el8bin.tar.xz el8bin/
mv confluent_el8bin.tar.xz ~/rpmbuild/SOURCES/
mv confluent_el9bin.tar.xz ~/rpmbuild/SOURCES/
rm -rf el9bin
rm -rf el8bin
rpmbuild -ba confluent_osdeploy.spec
-38
View File
@@ -1,38 +0,0 @@
cd $(dirname $0)
VERSION=`git describe|cut -d- -f 1`
NUMCOMMITS=`git describe|cut -d- -f 2`
if [ "$NUMCOMMITS" != "$VERSION" ]; then
LASTNUM=$(echo $VERSION|rev|cut -d . -f 1|rev)
LASTNUM=$((LASTNUM+1))
FIRSTPART=$(echo $VERSION|rev|cut -d . -f 2- |rev)
VERSION=${FIRSTPART}.${LASTNUM}
VERSION=$VERSION~dev$NUMCOMMITS+`git describe|cut -d- -f 3`
fi
sed -e "s/#VERSION#/$VERSION/" confluent_osdeploy-aarch64.spec.tmpl > confluent_osdeploy-aarch64.spec
cd ..
cp ../LICENSE .
tar Jcvf confluent_osdeploy.tar.xz confluent_osdeploy
mv confluent_osdeploy.tar.xz ~/rpmbuild/SOURCES/
cd -
mkdir -p el9bin/opt/confluent/bin
mkdir -p el9bin/stateless-bin
mkdir -p el8bin/opt/confluent/bin
mkdir -p el8bin/stateless-bin
podman run --privileged --rm -v $(pwd)/utils:/buildutils -i -t el9builder make -C /buildutils
cd utils
mv confluent_imginfo copernicus clortho autocons ../el9bin/opt/confluent/bin
mv start_root urlmount ../el9bin/stateless-bin/
cd ..
podman run --privileged --rm -v $(pwd)/utils:/buildutils -i -t el8builder make -C /buildutils
cd utils
mv confluent_imginfo copernicus clortho autocons ../el8bin/opt/confluent/bin
mv start_root urlmount ../el8bin/stateless-bin/
cd ..
tar Jcvf confluent_el9bin.tar.xz el9bin/
tar Jcvf confluent_el8bin.tar.xz el8bin/
mv confluent_el8bin.tar.xz ~/rpmbuild/SOURCES/
mv confluent_el9bin.tar.xz ~/rpmbuild/SOURCES/
rm -rf el9bin
rm -rf el8bin
podman run --privileged --rm -v $HOME:/root el8builder rpmbuild -ba /root/confluent/confluent_osdeploy/confluent_osdeploy-aarch64.spec
@@ -1,570 +0,0 @@
#!/usr/bin/python
try:
import http.client as client
except ImportError:
import httplib as client
import ctypes
import ctypes.util
import glob
import os
import select
import socket
import subprocess
import ssl
import sys
import struct
import time
import re
import json
class InvalidApiKey(Exception):
pass
def msg_align(len):
return (len + 3) & ~3
try:
_protoparm = ssl.PROTOCOL_TLS_CLIENT
except AttributeError:
_protoparm = ssl.PROTOCOL_SSLv23
cryptname = ctypes.util.find_library('crypt')
if not cryptname:
if os.path.exists('/lib64/libcrypt.so.2') or os.path.exists('/usr/lib64/libcrypt.so.2'):
cryptname = 'libcrypt.so.2'
else:
cryptname = 'libcrypt.so.1'
c_libcrypt = ctypes.CDLL(cryptname)
c_crypt = c_libcrypt.crypt
c_crypt.argtypes = (ctypes.c_char_p, ctypes.c_char_p)
c_crypt.restype = ctypes.c_char_p
def get_my_addresses():
nlhdrsz = struct.calcsize('IHHII')
ifaddrsz = struct.calcsize('BBBBI')
# RTM_GETADDR = 22
# nlmsghdr struct: u32 len, u16 type, u16 flags, u32 seq, u32 pid
nlhdr = struct.pack('IHHII', nlhdrsz + ifaddrsz, 22, 0x301, 0, 0)
# ifaddrmsg struct: u8 family, u8 prefixlen, u8 flags, u8 scope, u32 index
ifaddrmsg = struct.pack('BBBBI', 0, 0, 0, 0, 0)
s = socket.socket(socket.AF_NETLINK, socket.SOCK_RAW, socket.NETLINK_ROUTE)
s.bind((0, 0))
s.sendall(nlhdr + ifaddrmsg)
addrs = []
while True:
pdata = s.recv(65536)
v = memoryview(pdata)
if struct.unpack('H', v[4:6])[0] == 3: # netlink done message
break
while len(v):
length, typ = struct.unpack('IH', v[:6])
if typ == 20:
fam, plen, _, scope, ridx = struct.unpack('BBBBI', v[nlhdrsz:nlhdrsz+ifaddrsz])
if scope in (253, 0):
rta = v[nlhdrsz+ifaddrsz:length]
while len(rta):
rtalen, rtatyp = struct.unpack('HH', rta[:4])
if rtalen < 4:
break
if rtatyp == 1:
addrs.append((fam, rta[4:rtalen], plen, ridx))
rta = rta[msg_align(rtalen):]
v = v[msg_align(length):]
return addrs
def scan_confluents():
srvs = {}
s6 = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
s6.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 1)
s6.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
s6.bind(('::', 1900))
s4 = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s4.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
s4.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
s4.bind(('0.0.0.0', 1900))
doneidxs = set([])
msg = 'M-SEARCH * HTTP/1.1\r\nST: urn:xcat.org:service:confluent:'
with open('/etc/confluent/confluent.deploycfg') as dcfg:
for line in dcfg.read().split('\n'):
if line.startswith('confluent_uuid:'):
confluentuuid = line.split(': ')[1]
msg += '/confluentuuid=' + confluentuuid
break
try:
with open('/sys/devices/virtual/dmi/id/product_uuid') as uuidin:
msg += '/uuid=' + uuidin.read().strip()
except Exception:
pass
for addrf in glob.glob('/sys/class/net/*/address'):
with open(addrf) as addrin:
hwaddr = addrin.read().strip()
msg += '/mac=' + hwaddr
msg = msg.encode('utf8')
for addr in get_my_addresses():
if addr[0] == socket.AF_INET6:
if addr[-1] in doneidxs:
continue
try:
s6.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_MULTICAST_IF, addr[-1])
except TypeError:
s6.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_MULTICAST_IF, addr[-1].tobytes())
try:
s6.sendto(msg, ('ff02::c', 1900))
except OSError:
pass
doneidxs.add(addr[-1])
elif addr[0] == socket.AF_INET:
try:
s4.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_IF, addr[1])
except TypeError:
s4.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_IF, addr[1].tobytes())
try:
s4.sendto(msg, ('239.255.255.250', 1900))
except OSError:
pass
r = select.select((s4, s6), (), (), 4)
srvlist = []
if r:
r = r[0]
while r:
for s in r:
(rsp, peer) = s.recvfrom(9000)
rsp = rsp.split(b'\r\n')
current = None
for line in rsp:
if line.startswith(b'NODENAME: '):
current = {}
elif line.startswith(b'DEFAULTNET: 1'):
current['isdefault'] = True
elif line.startswith(b'MGTIFACE: '):
current['mgtiface'] = line.replace(b'MGTIFACE: ', b'').strip().decode('utf8')
if len(peer) > 2:
current['myidx'] = peer[-1]
currip = peer[0]
if currip.startswith('fe80::') and '%' not in currip:
currip = '{0}%{1}'.format(currip, peer[-1])
srvs[currip] = current
srvlist.append(currip)
r = select.select((s4, s6), (), (), 2)
if r:
r = r[0]
return srvlist, srvs
def get_net_apikey(nodename, mgr):
alpha = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789./'
newpass = ''.join([alpha[x >> 2] for x in bytearray(os.urandom(32))])
salt = '$5$' + ''.join([alpha[x >> 2] for x in bytearray(os.urandom(8))])
newpass = newpass.encode('utf8')
salt = salt.encode('utf8')
crypted = c_crypt(newpass, salt)
for addrinfo in socket.getaddrinfo(mgr, 13001, 0, socket.SOCK_STREAM):
try:
clisock = socket.socket(addrinfo[0], addrinfo[1])
clisock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
if addrinfo[0] == socket.AF_INET:
cliaddr = ('0.0.0.0', 302)
else:
cliaddr = ('::', 302)
clisock.bind(cliaddr)
clisock.settimeout(1)
clisock.connect(addrinfo[-1])
rsp = clisock.recv(8)
if rsp != b'\xc2\xd1-\xa8\x80\xd8j\xba':
raise Exception('Unrecognized credential banner')
hellostr = bytearray([1, len(nodename)]) + bytearray(nodename.encode('utf8')) + bytearray(b'\x00\x00')
clisock.send(hellostr)
rsp = bytearray(clisock.recv(2))
if not rsp:
continue
if rsp[0] == 128:
continue
if rsp[0] == 2:
echotoken = clisock.recv(rsp[1])
clisock.recv(2) # drain \x00\x00
clisock.send(bytes(bytearray([3, rsp[1]])))
clisock.send(echotoken)
clisock.send(bytes(bytearray([4, len(crypted)])))
clisock.send(crypted)
clisock.send(b'\x00\x00')
rsp = bytearray(clisock.recv(2))
if rsp[0] == 5:
return newpass.decode('utf8')
finally:
clisock.close()
return ''
def get_apikey(nodename, hosts, errout=None):
apikey = ""
if os.path.exists('/etc/confluent/confluent.apikey'):
apikey = open('/etc/confluent/confluent.apikey').read().strip()
if apikey:
return apikey
while not apikey:
for host in hosts:
try:
apikey = get_net_apikey(nodename, host)
except OSError:
apikey = None
if apikey:
break
else:
srvlist, _ = scan_confluents()
for host in srvlist:
try:
apikey = get_net_apikey(nodename, host)
except OSError:
apikey = None
if apikey:
break
if not apikey:
errmsg = "Failed getting API token, check deployment.apiarmed attribute on {}\n".format(nodename)
sys.stderr.write(errmsg)
if errout:
errout.write(errmsg)
time.sleep(10)
with open('/etc/confluent/confluent.apikey', 'w+') as apiout:
apiout.write(apikey)
apikey = apikey.strip()
os.chmod('/etc/confluent/confluent.apikey', 0o600)
return apikey
class HTTPSClient(client.HTTPConnection, object):
def __init__(self, usejson=False, port=443, host=None, errout=None, phmac=None, checkonly=False):
self.phmac = phmac
self.errout = None
if errout:
self.errout = open(errout, 'w')
self.errout.flush()
self.stdheaders = {}
mgtiface = None
if usejson:
self.stdheaders['ACCEPT'] = 'application/json'
if host:
self.hosts = [host]
with open('/etc/confluent/confluent.info') as cinfo:
info = cinfo.read().split('\n')
for line in info:
if line.startswith('NODENAME:'):
node = line.split(' ')[1]
self.stdheaders['CONFLUENT_NODENAME'] = node
else:
self.hosts = []
info = open('/etc/confluent/confluent.info').read().split('\n')
havedefault = '0'
plainhost = ''
for line in info:
host = ''
if line.startswith('NODENAME:'):
node = line.split(' ')[1]
self.stdheaders['CONFLUENT_NODENAME'] = node
if line.startswith('MANAGER:') and not host:
host = line.split(' ')[1]
self.hosts.append(host)
if not plainhost:
plainhost = host
if line.startswith('EXTMGRINFO:'):
extinfo = line.split(' ')[1]
extinfo = extinfo.split('|')
if havedefault == '0' and extinfo[2] == '1':
host, mgtiface, havedefault = extinfo[:3]
if '%' in host:
ifidx = host.split('%', 1)[1]
with open('/tmp/confluent.ifidx', 'w+') as ifout:
ifout.write(ifidx)
if not mgtiface:
host, mgtiface, havedefault = extinfo[:3]
if host:
if havedefault == '0':
if '%' in host:
ifidx = host.split('%', 1)[1]
with open('/tmp/confluent.ifidx', 'w+') as ifout:
ifout.write(ifidx)
self.hosts.append(host)
try:
info = open('/etc/confluent/confluent.deploycfg').read().split('\n')
except Exception:
info = None
if info:
for line in info:
if line.startswith('deploy_server: ') or line.startswith('deploy_server_v6: '):
self.hosts.append(line.split(': ', 1)[1])
if plainhost and not self.hosts:
self.hosts.append(plainhost)
if self.phmac:
with open(phmac, 'r') as hmacin:
self.stdheaders['CONFLUENT_CRYPTHMAC'] = hmacin.read()
elif not checkonly:
self.stdheaders['CONFLUENT_APIKEY'] = get_apikey(node, self.hosts, errout=self.errout)
if mgtiface:
self.stdheaders['CONFLUENT_MGTIFACE'] = mgtiface
self.port = port
self.host = None
self.node = node
host = self.check_connections()
client.HTTPConnection.__init__(self, host, port)
self.connect()
def set_header(self, key, val):
self.stdheaders[key] = val
def check_connections(self):
foundsrv = None
hosts = self.hosts
ctx = ssl.SSLContext(_protoparm)
ctx.load_verify_locations('/etc/confluent/ca.pem')
ctx.verify_mode = ssl.CERT_REQUIRED
ctx.check_hostname = True
for timeo in (0.1, 5):
for host in hosts:
try:
addrinf = socket.getaddrinfo(host, self.port)[0]
psock = socket.socket(addrinf[0])
psock.settimeout(timeo)
psock.connect(addrinf[4])
chost = host.split('%', 1)[0]
ctx.wrap_socket(psock, server_hostname=chost)
foundsrv = host
psock.close()
break
except (OSError, socket.error):
continue
except ssl.SSLError:
continue
except ssl.CertificateError:
continue
else:
continue
break
if not foundsrv:
srvlist, srvs = scan_confluents()
hosts = []
for srv in srvlist:
if srvs[srv].get('isdefault', False):
hosts = [srv] + hosts
else:
hosts = hosts + [srv]
for host in hosts:
try:
addrinf = socket.getaddrinfo(host, self.port)[0]
psock = socket.socket(addrinf[0])
psock.settimeout(timeo)
psock.connect(addrinf[4])
foundsrv = host
psock.close()
break
except OSError:
continue
else:
raise Exception('Unable to reach any hosts')
return foundsrv
def connect(self, tries=3):
addrinf = socket.getaddrinfo(self.host, self.port)[0]
psock = socket.socket(addrinf[0])
psock.settimeout(15)
psock.connect(addrinf[4])
ctx = ssl.SSLContext(_protoparm)
ctx.load_verify_locations('/etc/confluent/ca.pem')
host = self.host.split('%', 1)[0]
if '[' not in host and ':' in host:
self.stdheaders['Host'] = '[{0}]'.format(host)
else:
self.stdheaders['Host'] = '{0}'.format(host)
ctx.verify_mode = ssl.CERT_REQUIRED
ctx.check_hostname = True
try:
self.sock = ctx.wrap_socket(psock, server_hostname=host)
except ssl.SSLError:
errmsg = 'Error validating certificate on deployer (try `osdeploy initialize -t` on the deployment server {0})\n'.format(host)
sys.stderr.write(errmsg)
if self.errout:
self.errout.write(errmsg)
self.errout.flush()
sys.exit(1)
except socket.timeout:
if not tries:
raise
return self.connect(tries=tries-1)
def grab_url(self, url, data=None, returnrsp=False):
return self.grab_url_with_status(url, data, returnrsp)[1]
def grab_url_with_status(self, url, data=None, returnrsp=False):
if data:
method = 'POST'
else:
method = 'GET'
authed = False
while not authed:
authed = True
self.request(method, url, data, headers=self.stdheaders)
rsp = self.getresponse()
if rsp.status >= 200 and rsp.status < 300:
if returnrsp:
return rsp.status, rsp
else:
return rsp.status, rsp.read()
if rsp.status == 401:
authed = False
rsp.read()
with open('/etc/confluent/confluent.apikey', 'w+') as akfile:
akfile.write('')
self.stdheaders['CONFLUENT_APIKEY'] = get_apikey(
self.node, [self.host], errout=self.errout)
raise Exception(rsp.read())
def get_current_vmnic_vswitch():
uplinkmatch = re.compile(r'^\s*Uplinks:\s*(.*)')
switchmatch = re.compile(r'^\s*Name:\s*(.*)')
vswinfo = subprocess.check_output(['localcli', 'network', 'vswitch', 'standard', 'list']).decode()
vmnic = None
vswitch_name = None
for info in vswinfo.split('\n'):
name_match = switchmatch.match(info)
if name_match:
vswitch_name = name_match.group(1).strip()
upinfo = uplinkmatch.match(info)
if upinfo:
vmnic = upinfo.group(1).strip()
if vmnic and 'vusb0' not in vmnic:
return vswitch_name, vmnic
return vswitch_name, vmnic
def get_available_nics():
nicinfo = subprocess.check_output(['localcli', 'network', 'nic', 'list']).decode('utf8').split('\n')
available_nics = {}
# Skip headers and separators
parsing_started = False
for line in nicinfo:
if re.match(r'^-+', line):
parsing_started = True
continue
if not parsing_started or not line.strip():
continue
parts = re.split(r'\s{2,}', line.strip())
if len(parts) >= 5:
nic_name = parts[0]
nic_status = parts[4] # "Link Status" este al 5-lea câmp
available_nics[nic_name] = nic_status
return available_nics
def is_esxi():
return os.path.isdir("/etc/vmware")
def fix_vswitch():
if is_esxi():
start_time = time.time()
while True:
current_vswitch, current_vmnic = get_current_vmnic_vswitch()
if current_vswitch is None:
raise RuntimeError("Panic: current vswitch is None")
if current_vmnic is None:
raise RuntimeError("Panic: current vmnic is None")
available_nics = get_available_nics()
if current_vmnic and available_nics.get(current_vmnic) == 'Up':
break
new_vmnic = next((nic for nic, status in available_nics.items() if status == 'Up'), None)
if new_vmnic and new_vmnic != current_vmnic:
subprocess.check_call(['localcli', 'network', 'vswitch', 'standard', 'uplink', 'remove',
'--uplink-name', current_vmnic, '--vswitch-name', current_vswitch])
subprocess.check_call(['localcli', 'network', 'vswitch', 'standard', 'uplink', 'add',
'--uplink-name', new_vmnic, '--vswitch-name', current_vswitch])
elif not new_vmnic:
if time.time() - start_time > 300:
break
time.sleep(5)
time.sleep(5)
if __name__ == '__main__':
data = None
if '-f' in sys.argv:
try:
fix_vswitch()
except Exception as e:
print(f"fix_vswitch() error: {e}")
sys.argv.remove('-f')
sys.exit(0)
usejson = False
if '-j' in sys.argv:
usejson = True
if len(sys.argv) == 1:
HTTPSClient()
sys.exit(0)
try:
outbin = sys.argv.index('-o')
sys.argv.pop(outbin)
outbin = sys.argv.pop(outbin)
except ValueError:
outbin = None
try:
waitfor = sys.argv.index('-w')
sys.argv.pop(waitfor)
waitfor = int(sys.argv.pop(waitfor))
except ValueError:
waitfor = None
try:
data = sys.argv.index('-d')
sys.argv.pop(data)
data = sys.argv.pop(data)
except ValueError:
data = None
try:
errout = sys.argv.index('-e')
sys.argv.pop(errout)
errout = sys.argv.pop(errout)
except ValueError:
errout = None
try:
phmac = sys.argv.index('-p')
sys.argv.pop(phmac)
phmac = sys.argv.pop(phmac)
except ValueError:
phmac = None
try:
checkonly = False
idxit = sys.argv.index('-c')
sys.argv.pop(idxit)
checkonly = True
except Exception:
pass
if len(sys.argv) > 2 and os.path.exists(sys.argv[-1]):
data = open(sys.argv[-1]).read()
if outbin:
with open(outbin, 'ab+') as outf:
reader = HTTPSClient(usejson=usejson, errout=errout).grab_url(
sys.argv[1], data, returnrsp=True)
chunk = reader.read(16384)
while chunk:
outf.write(chunk)
chunk = reader.read(16384)
sys.exit(0)
client = HTTPSClient(usejson, errout=errout, phmac=phmac, checkonly=checkonly)
if waitfor:
status = 201
while status != waitfor:
status, rsp = client.grab_url_with_status(sys.argv[1], data)
sys.stdout.write(rsp.decode())
elif checkonly:
sys.stdout.write(client.check_connections())
else:
sys.stdout.write(client.grab_url(sys.argv[1], data).decode())
@@ -1,117 +0,0 @@
#!/usr/bin/python3
# This script evaluates whether firmware redirection is likely. It uses three cues:
# - Does the system offer up SPCR? This would indicate that the firmware is doing serial output.
# Otherwise, there's no indication that the firmware cares about serial console.
# - Is the system EFI? BIOS implementations may not intercept text draw calls after POST exit,
# thus even when BIOS tells us serial port is in use, it may not be doing anything when
# grub would be running
# - Is the serial port connected? In the event that firmware indicates serial port, but
# serial port is not reporting DCD, then it doesn't look like a comfortable enough scenario
import fcntl
import os
import os.path
import struct
import subprocess
import termios
addrtoname = {
0x3f8: '/dev/ttyS0',
0x2f8: '/dev/ttyS1',
0x3e8: '/dev/ttyS2',
0x2e8: '/dev/ttyS3',
}
speedmap = {
0: None,
3: 9600,
4: 19200,
6: 57600,
7: 115200,
}
termiobaud = {
9600: termios.B9600,
19200: termios.B19200,
57600: termios.B57600,
115200: termios.B115200,
}
def deserialize_grub_rh():
if 'console=ttyS' in open('/proc/cmdline').read():
return None # User manually indicated serial config
# they own the grub behavior too for now
grublines = []
with open('/etc/default/grub') as grubin:
grublines = grubin.read().split('\n')
with open('/etc/default/grub', 'w') as grubout:
for grubline in grublines:
if grubline.startswith('GRUB_TERMINAL'):
grubline = grubline.replace('serial ', '')
grubout.write(grubline + '\n')
subprocess.check_call(['grub2-mkconfig', '-o', '/boot/grub2/grub.cfg'])
def fixup_ubuntu_grub_serial():
# Ubuntu aggressively tries to graphics up
# grub. We will counter that for serial
# They also aggressively hide UI and
# block ability to interject. We will
# compromise and lean on nodeboot <node> setup
# as a means to give someone reasonable shot at
# the short timeout
with open('/etc/default/grub') as grubin:
grublines = grubin.read().split('\n')
with open('/etc/default/grub', 'w') as grubout:
for grubline in grublines:
if grubline.startswith('GRUB_TIMEOUT_STYLE=hidden'):
grubline = 'GRUB_TIMEOUT_STYLE=menu'
elif grubline.startswith('GRUB_TIMEOUT=0'):
grubline = 'GRUB_TIMEOUT=2'
elif grubline.startswith('#GRUB_TERMINAL=console'):
grubline = grubline.replace('#', '')
grubout.write(grubline + '\n')
subprocess.check_call(['update-grub'])
def get_serial_config():
if not os.path.exists('/sys/firmware/efi'):
return None
spcr = open("/sys/firmware/acpi/tables/SPCR", "rb")
spcr = bytearray(spcr.read())
if spcr[8] != 2 or spcr[36] != 0 or spcr[40] != 1:
return None
address = struct.unpack('<Q', spcr[44:52])[0]
tty = None
try:
tty = addrtoname[address]
except KeyError:
return None
retval = { 'tty': tty }
try:
retval['speed'] = speedmap[spcr[58]]
except KeyError:
return None
if retval['speed']:
ttyf = os.open(tty, os.O_RDWR | os.O_NOCTTY)
currattr = termios.tcgetattr(ttyf)
currattr[4:6] = [0, termiobaud[retval['speed']]]
termios.tcsetattr(ttyf, termios.TCSANOW, currattr)
retval['connected'] = bool(struct.unpack('<I', fcntl.ioctl(
ttyf, termios.TIOCMGET, '\x00\x00\x00\x00'))[0] & termios.TIOCM_CAR)
os.close(ttyf)
return retval
def main():
autoconscfg = get_serial_config()
if not autoconscfg or not autoconscfg['connected']:
return
if os.path.exists('/etc/redhat-release'): # redhat family
deserialize_grub_rh()
elif os.path.exists('/etc/os-release'):
with open('/etc/os-release') as osr:
if 'Ubuntu' in osr.read():
fixup_ubuntu_grub_serial()
if __name__ == '__main__':
main()
@@ -1,565 +0,0 @@
#!/usr/bin/python
import glob
import json
import os
import socket
import sys
import time
import shlex
import subprocess
try:
import yaml
except ImportError:
pass
try:
from importlib.machinery import SourceFileLoader
def load_source(mod, path):
return SourceFileLoader(mod, path).load_module()
except ImportError:
from imp import load_source
try:
apiclient = load_source('apiclient', '/opt/confluent/bin/apiclient')
except IOError:
apiclient = load_source('apiclient', '/etc/confluent/apiclient')
def add_lla(iface, mac):
pieces = mac.split(':')
initbyte = int(pieces[0], 16) ^ 2
lla = 'fe80::{0:x}{1}:{2}ff:fe{3}:{4}{5}/64'.format(initbyte, pieces[1], pieces[2], pieces[3], pieces[4], pieces[5])
try:
with open('/proc/sys/net/ipv6/conf/{0}/disable_ipv6'.format(iface), 'w') as setin:
setin.write('0')
subprocess.check_call(['ip', 'addr', 'add', 'dev', iface, lla, 'scope', 'link'])
except Exception:
return None
return lla
#cli = apiclient.HTTPSClient(json=True)
#c = cli.grab_url_with_status('/confluent-api/self/netcfg')
def add_missing_llas():
#NetworkManager goes out of its way to suppress ipv6 lla, so will just add some
added = {}
linkinfo = subprocess.check_output(['ip', '-br', 'l']).decode('utf8')
ifaces = {}
for line in linkinfo.split('\n'):
line = line.strip().split()
if not line or 'LOOPBACK' in line[-1] or 'NO-CARRIER' in line[-1]:
continue
if 'UP' not in line[-1]:
subprocess.call(['ip', 'link', 'set', line[0], 'up'])
ifaces[line[0]] = line[2]
ips = {}
ipinfo = subprocess.check_output(['ip', '-br', '-6', 'a']).decode('utf8')
for line in ipinfo.split('\n'):
line = line.strip().split(None, 2)
if not line:
continue
ips[line[0]] = line[2]
for iface in ifaces:
for addr in ips.get(iface, '').split():
if addr.startswith('fe80::'):
break
else:
newlla = add_lla(iface, ifaces[iface])
if newlla:
added[iface] = newlla
return added
def rm_tmp_llas(tmpllas):
for iface in tmpllas:
subprocess.check_call(['ip', 'addr', 'del', 'dev', iface, tmpllas[iface]])
def await_tentative():
maxwait = 10
while b'tentative' in subprocess.check_output(['ip', 'a']):
if maxwait == 0:
break
maxwait -= 1
time.sleep(1)
def map_idx_to_name():
map = {}
devtype = {}
prevdev = None
for line in subprocess.check_output(['ip', 'l']).decode('utf8').splitlines():
if line.startswith(' ') and 'link/' in line:
typ = line.split()[0].split('/')[1]
devtype[prevdev] = typ if typ != 'ether' else 'ethernet'
if line.startswith(' '):
continue
idx, iface, rst = line.split(':', 2)
prevdev = iface.strip()
rst = rst.split()
try:
midx = rst.index('master')
continue
except ValueError:
pass
idx = int(idx)
iface = iface.strip()
map[idx] = iface
return map, devtype
def get_interface_name(iname, settings):
explicitname = settings.get('interface_names', None)
if explicitname:
return explicitname
if settings.get('current_nic', False):
return iname
return None
class NetplanManager(object):
def __init__(self, deploycfg):
self.cfgbydev = {}
self.read_connections()
self.deploycfg = deploycfg
def read_connections(self):
for plan in glob.glob('/etc/netplan/*.y*ml'):
with open(plan) as planfile:
planinfo = yaml.safe_load(planfile)
if not planinfo:
continue
nicinfo = planinfo.get('network', {}).get('ethernets', {})
for devname in nicinfo:
if devname == 'lo':
continue
if 'gateway4' in nicinfo[devname]:
# normalize deprecated syntax on read in
gw4 = nicinfo[devname]['gateway4']
del nicinfo[devname]['gateway4']
routeinfo = nicinfo[devname].get('routes', [])
for ri in routeinfo:
if ri.get('via', None) == gw4 and ri.get('to', None) in ('default', '0.0.0.0/0', '0/0'):
break
else:
routeinfo.append({
'to': 'default',
'via': gw4
})
nicinfo[devname]['routes'] = routeinfo
self.cfgbydev[devname] = nicinfo[devname]
def apply_configuration(self, cfg):
devnames = cfg['interfaces']
if len(devnames) != 1:
raise Exception('Multi-nic team/bonds not yet supported')
stgs = cfg['settings']
needcfgapply = False
for devname in devnames:
needcfgwrite = False
# ipv6_method missing at uconn...
if stgs.get('ipv6_method', None) == 'static':
curraddr = stgs['ipv6_address']
currips = self.getcfgarrpath([devname, 'addresses'])
if curraddr not in currips:
needcfgwrite = True
currips.append(curraddr)
if stgs.get('ipv4_method', None) == 'static':
curraddr = stgs['ipv4_address']
currips = self.getcfgarrpath([devname, 'addresses'])
if curraddr not in currips:
needcfgwrite = True
currips.append(curraddr)
gws = []
gws.append(stgs.get('ipv4_gateway', None))
gws.append(stgs.get('ipv6_gateway', None))
for gwaddr in gws:
if gwaddr:
cfgroutes = self.getcfgarrpath([devname, 'routes'])
for rinfo in cfgroutes:
if rinfo.get('via', None) == gwaddr:
break
else:
needcfgwrite = True
cfgroutes.append({'via': gwaddr, 'to': 'default'})
dnsips = self.deploycfg.get('nameservers', [])
dnsdomain = self.deploycfg.get('dnsdomain', '')
if dnsips:
currdnsips = self.getcfgarrpath([devname, 'nameservers', 'addresses'])
for dnsip in dnsips:
if dnsip and dnsip not in currdnsips:
needcfgwrite = True
currdnsips.append(dnsip)
if dnsdomain:
currdnsdomain = self.getcfgarrpath([devname, 'nameservers', 'search'])
if dnsdomain not in currdnsdomain:
needcfgwrite = True
currdnsdomain.append(dnsdomain)
if needcfgwrite:
needcfgapply = True
newcfg = {'network': {'version': 2, 'ethernets': {devname: self.cfgbydev[devname]}}}
oumask = os.umask(0o77)
with open('/etc/netplan/{0}-confluentcfg.yaml'.format(devname), 'w') as planout:
planout.write(yaml.dump(newcfg))
os.umask(oumask)
if needcfgapply:
subprocess.call(['netplan', 'apply'])
def getcfgarrpath(self, devpath):
currptr = self.cfgbydev
for k in devpath[:-1]:
if k not in currptr:
currptr[k] = {}
currptr = currptr[k]
if devpath[-1] not in currptr:
currptr[devpath[-1]] = []
return currptr[devpath[-1]]
class WickedManager(object):
def __init__(self):
self.teamidx = 0
self.read_connections()
def read_connections(self):
self.cfgbydev = {}
for ifcfg in glob.glob('/etc/sysconfig/network/ifcfg-*'):
devname = ifcfg.replace('/etc/sysconfig/network/ifcfg-', '')
if devname == 'lo':
continue
currcfg = {}
self.cfgbydev[devname] = currcfg
for cfg in open(ifcfg).read().splitlines():
cfg = cfg.split('#', 1)[0]
try:
kv = ' '.join(shlex.split(cfg)).split('=', 1)
except Exception:
# unparseable line, likely having something we can't handle
del self.cfgbydev[devname]
if len(kv) != 2:
continue
k, v = kv
k = k.strip()
v = v.strip()
currcfg[k] = v
def apply_configuration(self, cfg):
stgs = cfg['settings']
ipcfg = 'STARTMODE=auto\n'
routecfg = ''
bootproto4 = stgs.get('ipv4_method', 'none')
bootproto6 = stgs.get('ipv6_method', 'none')
if bootproto4 == 'dhcp' and bootproto6 == 'dhcp':
ipcfg += 'BOOTPROTO=dhcp\n'
elif bootproto4 == 'dhcp':
ipcfg += 'BOOTPROTO=dhcp4\n'
elif bootproto6 == 'dhcp':
ipcfg += 'BOOTPROTO=dhcp6\n'
else:
ipcfg += 'BOOTPROTO=static\n'
if stgs.get('ipv4_address', None):
ipcfg += 'IPADDR=' + stgs['ipv4_address'] + '\n'
v4gw = stgs.get('ipv4_gateway', None)
if stgs.get('ipv6_address', None):
ipcfg += 'IPADDR_V6=' + stgs['ipv6_address'] + '\n'
v6gw = stgs.get('ipv6_gateway', None)
cname = None
if len(cfg['interfaces']) > 1: # creating new team
if not stgs.get('team_mode', None):
sys.stderr.write("Warning, multiple interfaces ({0}) without a team_mode, skipping setup\n".format(','.join(cfg['interfaces'])))
return
if not stgs.get('connection_name', None):
stgs['connection_name'] = 'bond{0}'.format(self.teamidx)
self.teamidx += 1
cname = stgs['connection_name']
with open('/etc/sysconfig/network/ifcfg-{0}'.format(cname), 'w') as teamout:
teamout.write(ipcfg)
if stgs['team_mode'] == 'lacp':
stgs['team_mode'] = '802.3ad'
teamout.write("BONDING_MODULE_OPTS='mode={0} miimon=100'\nBONDING_MASTER=yes\n".format(stgs['team_mode']))
idx = 1
for iface in cfg['interfaces']:
subprocess.call(['wicked', 'ifdown', iface])
try:
os.remove('/etc/sysconfig/network/ifcfg-{0}'.format(iface))
os.remove('/etc/sysconfig/network/ifroute-{0}'.format(iface))
except OSError:
pass
teamout.write('BONDING_SLAVE{0}={1}\n'.format(idx, iface))
idx += 1
else:
cname = list(cfg['interfaces'])[0]
priorcfg = self.cfgbydev.get(cname, {})
for cf in priorcfg:
if cf.startswith('TEAM_'):
ipcfg += '{0}={1}\n'.format(cf, priorcfg[cf])
with open('/etc/sysconfig/network/ifcfg-{0}'.format(cname), 'w') as iout:
iout.write(ipcfg)
if v4gw:
routecfg += 'default {0} - {1}\n'.format(v4gw, cname)
if v6gw:
routecfg += 'default {0} - {1}\n'.format(v6gw, cname)
if routecfg:
with open('/etc/sysconfig/network/ifroute-{0}'.format(cname), 'w') as routeout:
routeout.write(routecfg)
subprocess.call(['wicked', 'ifup', cname])
class NetworkManager(object):
bondtypes = {
'lacp': '802.3ad',
'loadbalance': 'balance-alb',
'roundrobin': 'balance-rr',
'activebackup': 'active-backup',
}
def __init__(self, devtypes, deploycfg):
self.deploycfg = deploycfg
self.connections = {}
self.uuidbyname = {}
self.uuidbydev = {}
self.connectiondetail = {}
self.read_connections()
self.teamidx = 0
self.devtypes = devtypes
def read_connections(self):
self.connections = {}
self.uuidbyname = {}
self.uuidbydev = {}
self.connectiondetail = {}
ci = subprocess.check_output(['nmcli', '-t', 'c']).decode('utf8')
for inf in ci.splitlines():
n, u, t, dev = inf.split(':')
if n == 'NAME':
continue
if dev == '--':
dev = None
self.uuidbyname[n] = u
if dev:
self.uuidbydev[dev] = u
self.connections[u] = {'name': n, 'uuid': u, 'type': t, 'dev': dev}
deats = {}
for deat in subprocess.check_output(['nmcli', 'c', 's', u]).decode('utf8').splitlines():
k, v = deat.split(':', 1)
v = v.strip()
if v == '--':
continue
if '(default)' in v:
continue
deats[k] = v
self.connectiondetail[u] = deats
def add_team_member(self, team, member):
bondcfg = {}
if member in self.uuidbydev:
myuuid = self.uuidbydev[member]
deats = self.connectiondetail[myuuid]
currteam = deats.get('connection.master', None)
if currteam == team:
return
for stg in ('ipv4.dhcp-hostname', 'ipv4.dns', 'ipv6.dns', 'ipv6.dhcp-hostname'):
if deats.get(stg, None):
bondcfg[stg] = deats[stg]
if member in self.uuidbyname:
subprocess.check_call(['nmcli', 'c', 'del', self.uuidbyname[member]])
devtype = self.devtypes.get(member, 'bond-slave')
subprocess.check_call(['nmcli', 'c', 'add', 'type', devtype, 'master', team, 'con-name', member, 'connection.interface-name', member])
if bondcfg:
args = []
for parm in bondcfg:
args.append(parm)
args.append(bondcfg[parm])
subprocess.check_call(['nmcli', 'c', 'm', team] + args)
def apply_configuration(self, cfg, lastchance=False):
cmdargs = {}
cmdargs['connection.autoconnect'] = 'yes'
stgs = cfg['settings']
cmdargs['ipv6.method'] = stgs.get('ipv6_method', 'link-local')
if stgs.get('ipv6_address', None):
cmdargs['ipv6.addresses'] = stgs['ipv6_address']
cmdargs['ipv4.method'] = stgs.get('ipv4_method', 'disabled')
if stgs.get('ipv4_address', None):
cmdargs['ipv4.addresses'] = stgs['ipv4_address']
if stgs.get('ipv4_gateway', None):
cmdargs['ipv4.gateway'] = stgs['ipv4_gateway']
if stgs.get('ipv6_gateway', None):
cmdargs['ipv6.gateway'] = stgs['ipv6_gateway']
dnsips = self.deploycfg.get('nameservers', [])
if not dnsips:
dnsips = []
dns4 = []
dns6 = []
for dnsip in dnsips:
if '.' in dnsip:
dns4.append(dnsip)
elif ':' in dnsip:
dns6.append(dnsip)
if dns4:
cmdargs['ipv4.dns'] = ','.join(dns4)
if dns6:
cmdargs['ipv6.dns'] = ','.join(dns6)
if len(cfg['interfaces']) > 1: # team time.. should be..
if not cfg['settings'].get('team_mode', None):
sys.stderr.write("Warning, multiple interfaces ({0}) without a team_mode, skipping setup\n".format(','.join(cfg['interfaces'])))
return
if not cfg['settings'].get('connection_name', None):
cfg['settings']['connection_name'] = 'team{0}'.format(self.teamidx)
self.teamidx += 1
cname = cfg['settings']['connection_name']
cargs = []
for arg in cmdargs:
cargs.append(arg)
cargs.append(cmdargs[arg])
if stgs['team_mode'] in self.bondtypes:
stgs['team_mode'] = self.bondtypes[stgs['team_mode']]
subprocess.check_call(['nmcli', 'c', 'add', 'type', 'bond', 'con-name', cname, 'connection.interface-name', cname, 'bond.options', 'miimon=100,mode={}'.format(stgs['team_mode'])] + cargs)
for iface in cfg['interfaces']:
self.add_team_member(cname, iface)
subprocess.check_call(['nmcli', 'c', 'u', cname])
else:
cname = stgs.get('connection_name', None)
iname = list(cfg['interfaces'])[0]
ctype = self.devtypes.get(iname, None)
if not ctype:
if lastchance:
sys.stderr.write("Warning, no device found for interface_name ({0}), skipping setup\n".format(iname))
return 1
if stgs.get('vlan_id', None):
vlan = stgs['vlan_id']
if ctype == 'infiniband':
vlan = '0x{0}'.format(vlan) if not vlan.startswith('0x') else vlan
cmdargs['infiniband.parent'] = iname
cmdargs['infiniband.p-key'] = vlan
iname = '{0}.{1}'.format(iname, vlan[2:])
elif ctype == 'ethernet':
ctype = 'vlan'
cmdargs['vlan.parent'] = iname
cmdargs['vlan.id'] = vlan
iname = '{0}.{1}'.format(iname, vlan)
else:
sys.stderr.write("Warning, unknown interface_name ({0}) device type ({1}) for VLAN/PKEY, skipping setup\n".format(iname, ctype))
return
cname = iname if not cname else cname
u = self.uuidbyname.get(cname, None)
cargs = []
for arg in cmdargs:
cargs.append(arg)
cargs.append(cmdargs[arg])
if u:
subprocess.check_call(['nmcli', 'c', 'm', u, 'connection.interface-name', iname] + cargs)
subprocess.check_call(['nmcli', 'c', 'u', u])
else:
subprocess.check_call(['nmcli', 'c', 'add', 'type', ctype, 'con-name', cname, 'connection.interface-name', iname] + cargs)
self.read_connections()
u = self.uuidbyname.get(cname, None)
if u:
subprocess.check_call(['nmcli', 'c', 'u', u])
if __name__ == '__main__':
havefirewall = subprocess.call(['systemctl', 'status', 'firewalld'])
havefirewall = havefirewall == 0
if havefirewall:
subprocess.check_call(['systemctl', 'stop', 'firewalld'])
tmpllas = add_missing_llas()
await_tentative()
idxmap, devtypes = map_idx_to_name()
netname_to_interfaces = {}
myaddrs = apiclient.get_my_addresses()
srvs, _ = apiclient.scan_confluents()
doneidxs = set([])
dc = None
if not srvs: # the multicast scan failed, fallback to deploycfg cfg file
with open('/etc/confluent/confluent.deploycfg', 'r') as dci:
for cfgline in dci.read().split('\n'):
if cfgline.startswith('deploy_server:'):
srvs = [cfgline.split()[1]]
break
for srv in srvs:
try:
s = socket.create_connection((srv, 443))
except socket.error:
continue
myname = s.getsockname()
s.close()
if len(myname) == 4:
curridx = myname[-1]
else:
myname = myname[0]
myname = socket.inet_pton(socket.AF_INET, myname)
for addr in myaddrs:
if myname == addr[1].tobytes():
curridx = addr[-1]
if curridx in doneidxs:
continue
for tries in (1, 2, 3):
try:
status, nc = apiclient.HTTPSClient(usejson=True, host=srv).grab_url_with_status('/confluent-api/self/netcfg')
break
except Exception:
if tries == 3:
raise
time.sleep(1)
continue
nc = json.loads(nc)
if not dc:
for tries in (1, 2, 3):
try:
status, dc = apiclient.HTTPSClient(usejson=True, host=srv).grab_url_with_status('/confluent-api/self/deploycfg2')
break
except Exception:
if tries == 3:
raise
time.sleep(1)
continue
dc = json.loads(dc)
iname = get_interface_name(idxmap[curridx], nc.get('default', {}))
if iname:
for iname in iname.split(','):
if 'default' in netname_to_interfaces:
netname_to_interfaces['default']['interfaces'].add(iname)
else:
netname_to_interfaces['default'] = {'interfaces': set([iname]), 'settings': nc['default']}
for netname in nc.get('extranets', {}):
uname = '_' + netname
iname = get_interface_name(idxmap[curridx], nc['extranets'][netname])
if iname:
for iname in iname.split(','):
if uname in netname_to_interfaces:
netname_to_interfaces[uname]['interfaces'].add(iname)
else:
netname_to_interfaces[uname] = {'interfaces': set([iname]), 'settings': nc['extranets'][netname]}
doneidxs.add(curridx)
if 'default' in netname_to_interfaces:
for netn in netname_to_interfaces:
if netn == 'default':
continue
netname_to_interfaces['default']['interfaces'] -= netname_to_interfaces[netn]['interfaces']
if not netname_to_interfaces['default']['interfaces']:
del netname_to_interfaces['default']
# Make sure VLAN/PKEY connections are created last
netname_to_interfaces = dict(sorted(netname_to_interfaces.items(), key=lambda item: 'vlan_id' in item[1]['settings']))
rm_tmp_llas(tmpllas)
if os.path.exists('/usr/sbin/netplan'):
nm = NetplanManager(dc)
if os.path.exists('/usr/bin/nmcli'):
nm = NetworkManager(devtypes, dc)
elif os.path.exists('/usr/sbin/wicked'):
nm = WickedManager()
retrynics = []
for netn in netname_to_interfaces:
redo = nm.apply_configuration(netname_to_interfaces[netn])
if redo == 1:
retrynics.append(netn)
if retrynics:
idxmap, devtypes = map_idx_to_name()
if os.path.exists('/usr/sbin/netplan'):
nm = NetplanManager(dc)
if os.path.exists('/usr/bin/nmcli'):
nm = NetworkManager(devtypes, dc)
elif os.path.exists('/usr/sbin/wicked'):
nm = WickedManager()
for netn in retrynics:
nm.apply_configuration(netname_to_interfaces[netn], lastchance=True)
if havefirewall:
subprocess.check_call(['systemctl', 'start', 'firewalld'])
await_tentative()
@@ -1,43 +0,0 @@
[ -f /lib/confluent/functions ] && . /lib/confluent/functions
[ -f /etc/confluent/functions ] && . /etc/confluent/functions
[ -f /opt/confluent/bin/apiclient ] && confapiclient=/opt/confluent/bin/apiclient
[ -f /etc/confluent/apiclient ] && confapiclient=/etc/confluent/apiclient
for pubkey in /etc/ssh/ssh_host*key.pub; do
if [ "$pubkey" = /etc/ssh/ssh_host_key.pub ]; then
continue
fi
certfile=${pubkey/.pub/-cert.pub}
rm $certfile
confluentpython $confapiclient /confluent-api/self/sshcert $pubkey -o $certfile
done
if [ -d /etc/ssh/sshd_config.d/ -a ! -e /etc/ssh/sshd_config.d/90-confluent.conf ]; then
for cert in /etc/ssh/ssh*-cert.pub; do
echo HostCertificate $cert >> /etc/ssh/sshd_config.d/90-confluent.conf
done
echo HostbasedAuthentication yes >> /etc/ssh/sshd_config.d/90-confluent.conf
echo HostbasedUsesNameFromPacketOnly yes >> /etc/ssh/sshd_config.d/90-confluent.conf
echo IgnoreRhosts no >> /etc/ssh/sshd_config.d/90-confluent.conf
fi
TMPDIR=$(mktemp -d)
cd $TMPDIR
confluentpython $confapiclient /confluent-public/site/initramfs.tgz -o initramfs.tgz
tar xf initramfs.tgz
for ca in ssh/*.ca; do
LINE=$(cat $ca)
cp -af /etc/ssh/ssh_known_hosts /etc/ssh/ssh_known_hosts.new
grep -v "$LINE" /etc/ssh/ssh_known_hosts > /etc/ssh/ssh_known_hosts.new
echo '@cert-authority *' $LINE >> /etc/ssh/ssh_known_hosts.new
mv /etc/ssh/ssh_known_hosts.new /etc/ssh/ssh_known_hosts
done
for pubkey in ssh/*.*pubkey; do
LINE=$(cat $pubkey)
cp -af /root/.ssh/authorized_keys /root/.ssh/authorized_keys.new
grep -v "$LINE" /root/.ssh/authorized_keys > /root/.ssh/authorized_keys.new
echo "$LINE" >> /root/.ssh/authorized_keys.new
mv /root/.ssh/authorized_keys.new /root/.ssh/authorized_keys
done
confluentpython $confapiclient /confluent-api/self/nodelist | sed -e 's/^- //' > /etc/ssh/shosts.equiv
cat /etc/ssh/shosts.equiv > /root/.shosts
cd -
rm -rf $TMPDIR
@@ -1,91 +0,0 @@
Name: confluent_osdeploy-aarch64
Version: #VERSION#
Release: 1
Summary: OS Deployment support for confluent
License: Apache2
URL: https://hpc.lenovo.com/
Source0: confluent_osdeploy.tar.xz
Source1: confluent_el9bin.tar.xz
Source2: confluent_el8bin.tar.xz
BuildArch: noarch
Requires: confluent_ipxe mtools tar
BuildRoot: /tmp
%description
This contains support utilities for enabling deployment of aarch64 architecture systems
%define debug_package %{nil}
%prep
%setup -n confluent_osdeploy -a 2 -a 1
%build
mkdir -p opt/confluent/bin
mkdir -p stateless-bin
cp -a el8bin/* .
ln -s el8 el9
for os in rhvh4 el7 genesis el8 suse15 ubuntu20.04 ubuntu22.04 ubuntu24.04 coreos el9; do
mkdir ${os}out
cd ${os}out
if [ -d ../${os}bin ]; then
cp -a ../${os}bin/opt .
else
cp -a ../opt .
fi
cp -a ../${os}/initramfs/* .
cp -a ../common/initramfs/* .
find . | cpio -H newc -o > ../addons.cpio
mv ../addons.cpio .
cd ..
done
for os in el7 el8 suse15 el9 ubuntu20.04; do
mkdir ${os}disklessout
cd ${os}disklessout
if [ -d ../${os}bin ]; then
cp -a ../${os}bin/opt .
else
cp -a ../opt .
fi
cp -a ../${os}-diskless/initramfs/* .
cp -a ../common/initramfs/* .
if [ -d ../${os}bin ]; then
cp -a ../${os}bin/stateless-bin/* opt/confluent/bin
else
cp -a ../stateless-bin/* opt/confluent/bin
fi
find . | cpio -H newc -o > ../addons.cpio
mv ../addons.cpio .
cd ..
done
mkdir esxi7out
cd esxi7out
cp -a ../opt .
cp -a ../esxi7/initramfs/* .
cp -a ../common/initramfs/* .
chmod +x bin/* opt/confluent/bin/*
tar zcvf ../addons.tgz *
mv ../addons.tgz .
cd ..
cp -a esxi7out esxi6out
cp -a esxi7 esxi6
cp -a esxi7out esxi8out
cp -a esxi7 esxi8
%install
mkdir -p %{buildroot}/opt/confluent/share/licenses/confluent_osdeploy/
#cp LICENSE %{buildroot}/opt/confluent/share/licenses/confluent_osdeploy/
for os in rhvh4 el7 el8 el9 genesis suse15 ubuntu20.04 ubuntu22.04 ubuntu24.04 esxi6 esxi7 esxi8 coreos; do
mkdir -p %{buildroot}/opt/confluent/lib/osdeploy/$os/initramfs/aarch64/
cp ${os}out/addons.* %{buildroot}/opt/confluent/lib/osdeploy/$os/initramfs/aarch64/
if [ -d ${os}disklessout ]; then
mkdir -p %{buildroot}/opt/confluent/lib/osdeploy/${os}-diskless/initramfs/aarch64/
cp ${os}disklessout/addons.* %{buildroot}/opt/confluent/lib/osdeploy/${os}-diskless/initramfs/aarch64/
fi
done
find %{buildroot}/opt/confluent/lib/osdeploy/ -name .gitignore -exec rm -f {} +
%files
/opt/confluent/lib/osdeploy
#%license /opt/confluent/share/licenses/confluent_osdeploy/LICENSE
@@ -1,109 +0,0 @@
Name: confluent_osdeploy-x86_64
Version: #VERSION#
Release: 1
Summary: OS Deployment support for confluent
License: Apache2
URL: https://hpc.lenovo.com/
Source0: confluent_osdeploy.tar.xz
Source1: confluent_el9bin.tar.xz
Source2: confluent_el8bin.tar.xz
BuildArch: noarch
Requires: confluent_ipxe mtools tar
BuildRoot: /tmp
%description
This contains support utilities for enabling deployment of x86_64 architecture systems
%define debug_package %{nil}
%prep
%setup -n confluent_osdeploy -a 2 -a 1
%build
#cd utils
#make all
#cp confluent_imginfo copernicus clortho autocons ../opt/confluent/bin
#cp start_root urlmount ../stateless-bin/
#cd ..
ln -s el8 el9
cp -a el8 el10
mv el10/initramfs/usr el10/initramfs/var
for os in rhvh4 el7 genesis el8 suse15 ubuntu18.04 ubuntu20.04 ubuntu22.04 ubuntu24.04 coreos el9 el10; do
mkdir ${os}out
cd ${os}out
if [ -d ../${os}bin ]; then
cp -a ../${os}bin/opt .
elif [ $os = el10 ]; then
cp -a ../el9bin/opt .
else
cp -a ../el8bin/opt .
fi
cp -a ../${os}/initramfs/* .
cp -a ../common/initramfs/* .
find . | cpio -H newc -o > ../addons.cpio
mv ../addons.cpio .
cd ..
done
for os in el7 el8 suse15 el9 ubuntu20.04 ubuntu22.04 ubuntu24.04; do
mkdir ${os}disklessout
cd ${os}disklessout
if [ -d ../${os}bin ]; then
cp -a ../${os}bin/opt .
else
cp -a ../el8bin/opt .
fi
cp -a ../${os}-diskless/initramfs/* .
cp -a ../common/initramfs/* .
if [ -d ../${os}bin ]; then
cp -a ../${os}bin/stateless-bin/* opt/confluent/bin
else
cp -a ../el8bin/stateless-bin/* opt/confluent/bin
fi
find . | cpio -H newc -o > ../addons.cpio
mv ../addons.cpio .
cd ..
done
mkdir esxi7out
cd esxi7out
cp -a ../el8bin/opt .
cp -a ../esxi7/initramfs/* .
cp -a ../common/initramfs/* .
chmod +x bin/* opt/confluent/bin/*
tar zcvf ../addons.tgz *
mv ../addons.tgz .
cd ..
cp -a esxi7out esxi6out
cp -a esxi7 esxi6
cp -a esxi7out esxi8out
cp -a esxi7out esxi9out
cp -a esxi7 esxi8
cp -a esxi7 esxi9
%install
mkdir -p %{buildroot}/opt/confluent/share/licenses/confluent_osdeploy/
cp LICENSE %{buildroot}/opt/confluent/share/licenses/confluent_osdeploy/
for os in rhvh4 el7 el8 el9 el10 genesis suse15 ubuntu20.04 ubuntu18.04 ubuntu22.04 ubuntu24.04 esxi6 esxi7 esxi8 esxi9 coreos; do
mkdir -p %{buildroot}/opt/confluent/lib/osdeploy/$os/initramfs
mkdir -p %{buildroot}/opt/confluent/lib/osdeploy/$os/profiles
cp ${os}out/addons.* %{buildroot}/opt/confluent/lib/osdeploy/$os/initramfs
cp -a $os/profiles/* %{buildroot}/opt/confluent/lib/osdeploy/$os/profiles
for targ in %{buildroot}/opt/confluent/lib/osdeploy/$os/profiles/*; do
cp -a common/profile/* $targ
done
if [ -d ${os}disklessout ]; then
mkdir -p %{buildroot}/opt/confluent/lib/osdeploy/${os}-diskless/initramfs
mkdir -p %{buildroot}/opt/confluent/lib/osdeploy/${os}-diskless/profiles
cp ${os}disklessout/addons.* %{buildroot}/opt/confluent/lib/osdeploy/${os}-diskless/initramfs
cp -a ${os}-diskless/profiles/* %{buildroot}/opt/confluent/lib/osdeploy/$os-diskless/profiles
for targ in %{buildroot}/opt/confluent/lib/osdeploy/$os-diskless/profiles/*; do
cp -a common/profile/* $targ
done
fi
done
find %{buildroot}/opt/confluent/lib/osdeploy/ -name .gitignore -exec rm -f {} +
%files
/opt/confluent/lib/osdeploy
%license /opt/confluent/share/licenses/confluent_osdeploy/LICENSE
@@ -1 +0,0 @@
/usr/lib/systemd/system/confluent-rootfs.service
@@ -1,172 +0,0 @@
#!/bin/bash
TRIES=0
oum=$(umask)
umask 0077
mkdir -p /etc/confluent
echo -n > /etc/confluent/confluent.info
umask $oum
if [ -e /dev/disk/by-label/CNFLNT_IDNT ]; then
tmnt=$(mktemp -d)
tcfg=$(mktemp)
mount /dev/disk/by-label/CNFLNT_IDNT $tmnt
cd $tmnt
deploysrvs=$(sed -n '/^deploy_servers:/, /^[^-]/p' cnflnt.yml |grep ^-|sed -e 's/^- //'|grep -v :)
nodename=$(grep ^nodename: cnflnt.yml|awk '{print $2}')
sed -n '/^net_cfgs:/, /^[^- ]/{/^[^- ]/!p}' cnflnt.yml |sed -n '/^-/, /^-/{/^-/!p}'| sed -e 's/^[- ]*//'> $tcfg
autoconfigmethod=$(grep ^ipv4_method: $tcfg)
autoconfigmethod=${autoconfigmethod#ipv4_method: }
if [ "$autoconfigmethod" = "dhcp" ]; then
/usr/libexec/nm-initrd-generator ip=:dhcp
else
v4addr=$(grep ^ipv4_address: $tcfg)
v4addr=${v4addr#ipv4_address: }
v4addr=${v4addr%/*}
v4gw=$(grep ^ipv4_gateway: $tcfg)
v4gw=${v4gw#ipv4_gateway: }
if [ "$v4gw" = "null" ]; then
v4gw=""
fi
v4nm=$(grep ipv4_netmask: $tcfg)
v4nm=${v4nm#ipv4_netmask: }
/usr/libexec/nm-initrd-generator ip=$v4addr::$v4gw:$v4nm:$hostname::none
fi
NetworkManager --configure-and-quit=initrd --no-daemon
hmackeyfile=$(mktemp)
echo -n $(grep ^apitoken: cnflnt.yml|awk '{print $2}') > $hmackeyfile
cd -
umount $tmnt
passfile=$(mktemp)
passcrypt=$(mktemp)
hmacfile=$(mktemp)
ln -s /opt/confluent/bin/clortho /opt/confluent/bin/genpasshmac
/opt/confluent/bin/genpasshmac $passfile $passcrypt $hmacfile $hmackeyfile
for deployer in $deploysrvs; do
if curl -f -H "CONFLUENT_NODENAME: $nodename" -H "CONFLUENT_CRYPTHMAC: $(cat $hmacfile)" -d@$passcrypt -k https://$deployer/confluent-api/self/registerapikey; then
cp $passfile /etc/confluent/confluent.apikey
confluent_apikey=$(cat /etc/confluent/confluent.apikey)
curl -sf -H "CONFLUENT_NODENAME: $nodename" -H "CONFLUENT_APIKEY: $confluent_apikey" https://$deployer/confluent-api/self/deploycfg > /etc/confluent/confluent.deploycfg
curl -sf -H "CONFLUENT_NODENAME: $nodename" -H "CONFLUENT_APIKEY: $confluent_apikey" https://$deployer/confluent-api/self/profileprivate/pending/config.ign > /config.ign
[ -s /config.ign ] || rm /config.ign
confluent_profile=$(grep ^profile: /etc/confluent/confluent.deploycfg)
confluent_profile=${confluent_profile#profile: }
curl -sf https://$deployer/confluent-public/os/$confluent_profile/rootfs.img | rdcore stream-hash /etc/coreos-live-want-rootfs | bsdtar -xf - -C /
exit 0
fi
done
fi
cd /sys/class/net
echo "Searching for confluent deployment server...." > /dev/console
while ! grep ^NODE /etc/confluent/confluent.info; do
while ! grep ^EXTMGRINFO: /etc/confluent/confluent.info | awk -F'|' '{print $3}' | grep 1 >& /dev/null && [ "$TRIES" -lt 60 ]; do
TRIES=$((TRIES + 1))
for currif in *; do
ip link set $currif up
done
/opt/confluent/bin/copernicus -t > /etc/confluent/confluent.info
done
if ! grep ^NODE /etc/confluent/confluent.info; then
echo 'Current net config:' > /dev/console
ip -br a > /dev/console
exit 1
fi
done
echo "Found confluent deployment services on local network" > /dev/console
cd /
echo -n "" > /tmp/confluent.initq
# restart cmdline
echo -n "" > /etc/cmdline.d/01-confluent.conf
nodename=$(grep ^NODENAME /etc/confluent/confluent.info|awk '{print $2}')
cat /tls/*.pem > /etc/confluent/ca.pem
confluent_mgr=$(grep ^MANAGER: /etc/confluent/confluent.info|head -n 1 | awk '{print $2}')
echo "Connecting to confluent server: $confluent_mgr" > /dev/console
if [[ $confluent_mgr == *%* ]]; then
echo $confluent_mgr | awk -F% '{print $2}' > /tmp/confluent.ifidx
ifidx=$(cat /tmp/confluent.ifidx)
ifname=$(ip link |grep ^$ifidx:|awk '{print $2}')
ifname=${ifname%:}
fi
needseal=1
oldumask=$(umask)
umask 0077
while [ -z "$confluent_apikey" ]; do
/opt/confluent/bin/clortho $nodename $confluent_mgr > /etc/confluent/confluent.apikey
if grep ^SEALED: /etc/confluent/confluent.apikey > /dev/null; then
needseal=0
sed -e s/^SEALED:// /etc/confluent/confluent.apikey | clevis-decrypt-tpm2 > /etc/confluent/confluent.apikey.decrypt
mv /etc/confluent/confluent.apikey.decrypt /etc/confluent/confluent.apikey
fi
confluent_apikey=$(cat /etc/confluent/confluent.apikey)
if [ -z "$confluent_apikey" ]; then
echo "Unable to acquire node api key, no TPM2 sealed nor fresh token available, retrying..." > /dev/console
sleep 10
fi
done
if [[ $confluent_mgr == *:* ]]; then
confluent_mgr="[$confluent_mgr]"
fi
if [ $needseal == 1 ]; then
sealed=$(echo $confluent_apikey | clevis-encrypt-tpm2 {})
if [ ! -z "$sealed" ]; then
curl -sf -H "CONFLUENT_NODENAME: $nodename" -H "CONFLUENT_APIKEY: $confluent_apikey" -d $sealed https://$confluent_mgr/confluent-api/self/saveapikey
fi
fi
curl -sf -H "CONFLUENT_NODENAME: $nodename" -H "CONFLUENT_APIKEY: $confluent_apikey" https://$confluent_mgr/confluent-api/self/deploycfg > /etc/confluent/confluent.deploycfg
curl -sf -H "CONFLUENT_NODENAME: $nodename" -H "CONFLUENT_APIKEY: $confluent_apikey" https://$confluent_mgr/confluent-api/self/profileprivate/pending/config.ign > /config.ign
[ -s /config.ign ] || rm /config.ign
umask $oldumask
autoconfigmethod=$(grep ^ipv4_method: /etc/confluent/confluent.deploycfg)
autoconfigmethod=${autoconfigmethod#ipv4_method: }
confluent_profile=$(grep ^profile: /etc/confluent/confluent.deploycfg)
confluent_profile=${confluent_profile#profile: }
if [ "$autoconfigmethod" = "dhcp" ]; then
echo ip=$ifname:dhcp >> /etc/cmdline.d/01-confluent.conf
else
v4addr=$(grep ^ipv4_address: /etc/confluent/confluent.deploycfg)
v4addr=${v4addr#ipv4_address: }
v4gw=$(grep ^ipv4_gateway: /etc/confluent/confluent.deploycfg)
v4gw=${v4gw#ipv4_gateway: }
if [ "$v4gw" = "null" ]; then
v4gw=""
fi
v4nm=$(grep ipv4_netmask: /etc/confluent/confluent.deploycfg)
v4nm=${v4nm#ipv4_netmask: }
echo ip=$v4addr::$v4gw:$v4nm:$hostname:$ifname:none >> /etc/cmdline.d/01-confluent.conf
fi
nameserversec=0
while read -r entry; do
if [ $nameserversec = 1 ]; then
if [[ $entry == "-"* ]] && [[ $entry != "- ''" ]]; then
echo nameserver=${entry#- } >> /etc/cmdline.d/01-confluent.conf
continue
fi
fi
nameserversec=0
if [ "${entry%:*}" = "nameservers" ]; then
nameserversec=1
continue
fi
done < /etc/confluent/confluent.deploycfg
if [ -e /lib/nm-lib.sh ]; then
. /lib/nm-lib.sh
nm_generate_connections
sed -i 's/method=disabled/method=link-local/' /run/NetworkManager/system-connections/*.nmconnection
if [ -f /run/NetworkManager/system-connections/$ifname.nmconnection ]; then
rm /run/NetworkManager/system-connections/default_connection.nmconnection
fi
if [[ "$ifname" == ib* ]]; then
sed -i s/type=ethernet/type=infiniband/ /run/NetworkManager/system-connections/$ifname.nmconnection
if ! grep '\[infiniband\]' /run/NetworkManager/system-connections/$ifname.nmconnection > /dev/null; then
echo >> /run/NetworkManager/system-connections/$ifname.nmconnection
echo '[infiniband]' >> /run/NetworkManager/system-connections/$ifname.nmconnection
echo transport-mode=datagram >> /run/NetworkManager/system-connections/$ifname.nmconnection
fi
fi
fi
#cat /proc/cmdline /etc/cmdline.d/01-confluent.conf | tr '\n' ' ' > /run/fakecmdline
#chcon system_u:object_r:proc_t:s0 /run/fakecmdline
#mount -o bind /run/fakecmdline /proc/cmdline
curl -sf https://$confluent_mgr/confluent-public/os/$confluent_profile/rootfs.img | rdcore stream-hash /etc/coreos-live-want-rootfs | bsdtar -xf - -C /
@@ -1,10 +0,0 @@
cat /tls/*.0 >> /etc/pki/tls/certs/ca-bundle.crt
if ! grep console= /proc/cmdline >& /dev/null; then
autocons=$(/opt/confluent/bin/autocons)
if [ -n "$autocons" ]; then
echo console=$autocons |sed -e 's!/dev/!!' >> /tmp/01-autocons.conf
autocons=${autocons%,*}
echo $autocons > /tmp/01-autocons.devnode
echo "Detected firmware specified console at $(cat /tmp/01-autocons.conf)" > $autocons
fi
fi
@@ -1,93 +0,0 @@
#!/bin/sh
nodename=$(grep ^NODENAME: /etc/confluent/confluent.info | awk '{print $2}')
confluent_mgr=$(grep ^MANAGER: /etc/confluent/confluent.info| head -n 1| awk '{print $2}' | sed -e s/%/%25/)
if [[ $confluent_mgr = *:* ]]; then
confluent_mgr=[$confluent_mgr]
fi
mkdir -p /sysroot/var/home/core/.ssh/
chmod 700 /sysroot/var/home/core/.ssh
cat /ssh/*.rootpubkey >> /sysroot/var/home/core/.ssh/authorized_keys
chmod 600 /sysroot/var/home/core/.ssh/authorized_keys
cp -a /etc/confluent /sysroot/etc/confluent
mkdir -p /sysroot/opt/confluent/bin/
cat > /sysroot/opt/confluent/bin/setssh.sh << 'EOF'
#!/bin/sh
nodename=$(grep ^NODENAME: /etc/confluent/confluent.info | awk '{print $2}')
confluent_mgr=$(grep ^MANAGER: /etc/confluent/confluent.info| head -n 1| awk '{print $2}' | sed -e s/%/%25/)
if [[ $confluent_mgr = *:* ]]; then
confluent_mgr=[$confluent_mgr]
fi
for i in /etc/ssh/ssh_host*key.pub; do
certname=${i/.pub/-cert.pub}
curl --cacert /etc/confluent/ca.pem -sf -X POST -H "CONFLUENT_NODENAME: $nodename" -H "CONFLUENT_APIKEY: $(cat /etc/confluent/confluent.apikey)" -d @$i https://$confluent_mgr/confluent-api/self/sshcert > $certname
if ! grep HostKey ${i%.pub} /etc/ssh/sshd_config > /dev/null; then
echo HostKey ${i%.pub} >> /etc/ssh/sshd_config
fi
if ! grep HostCertificate $certname /etc/ssh/sshd_config > /dev/null; then
echo HostCertificate $certname >> /etc/ssh/sshd_config
fi
done
EOF
chmod 755 /sysroot/opt/confluent/bin/setssh.sh
chcon system_u:object_r:bin_t:s0 /sysroot/opt/confluent/bin/setssh.sh
cat > /sysroot/etc/systemd/system/confluent-ssh.service << EOF
[Service]
Type=oneshot
ExecStart=/opt/confluent/bin/setssh.sh
[Unit]
PartOf=sshd.service
Before=sshd.service
After=sshd-keygen.target
Wants=sshd-keygen.target
Requires=network-online.target
After=network-online.target
[Install]
WantedBy=sshd.service
EOF
mkdir -p /sysroot/etc/systemd/system/sshd.service.wants
ln -s /etc/systemd/system/confluent-ssh.service /sysroot/etc/systemd/system/sshd.service.wants/
ln -s /etc/systemd/system/confluent-onboot.service /sysroot/etc/systemd/system/multi-user.target.wants/
cat > /sysroot/opt/confluent/bin/onboot.sh << 'EOF'
#!/bin/sh
confluent_profile=$(grep ^profile: /etc/confluent/confluent.deploycfg |awk '{print $2}')
nodename=$(grep ^NODENAME: /etc/confluent/confluent.info | awk '{print $2}')
confluent_mgr=$(grep ^MANAGER: /etc/confluent/confluent.info| head -n 1| awk '{print $2}' | sed -e s/%/%25/)
if [[ $confluent_mgr = *:* ]]; then
confluent_mgr=[$confluent_mgr]
fi
curl -sf https://$confluent_mgr/confluent-public/os/${confluent_profile}/scripts/onboot.sh > /tmp/onboot.sh
[ -s /tmp/onboot.sh ] && . /tmp/onboot.sh
exit 0
EOF
chmod 755 /sysroot/opt/confluent/bin/onboot.sh
chcon system_u:object_r:bin_t:s0 /sysroot/opt/confluent/bin/onboot.sh
cat > /sysroot/etc/systemd/system/confluent-onboot.service << EOF
[Service]
Type=oneshot
ExecStart=/opt/confluent/bin/onboot.sh
[Unit]
Requires=sshd.service
After=sshd.service
Requires=network-online.target
After=network-online.target
EOF
chcon -h system_u:object_r:systemd_unit_file_t:s0 /sysroot/etc/systemd/system/confluent-ssh.service /sysroot/etc/systemd/system/sshd.service.wants/confluent-ssh.service /sysroot/etc/systemd/system/confluent-onboot.service /sysroot/etc/systemd/system/multi-user.target.wants/confluent-onboot.service
cp -a /opt/confluent/bin/* /sysroot/opt/confluent/bin/
chcon system_u:object_r:bin_t:s0 /sysroot/opt/confluent/bin/*
@@ -1,22 +0,0 @@
[Unit]
Description=Confluent initialization
DefaultDependencies=false
After=basic.target
# Network is enabled here
After=dracut-initqueue.service
Before=ignition-setup-user.service
Before=coreos-ignition-setup-user.service
Before=coreos-livepxe-rootfs.service
# If we fail, the boot will fail. Be explicit about it.
OnFailure=emergency.target
OnFailureJobMode=isolate
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/opt/confluent/bin/initconfluent.sh
@@ -1,14 +0,0 @@
#!/bin/sh
profname=$(basename $2)
if file -L $1/images/ignition.img | grep gzip > /dev/null; then
mkdir -p /var/lib/confluent/private/os/${profname}/pending/
cd /var/lib/confluent/private/os/${profname}/pending/
zcat $1/images/ignition.img | cpio -dumiv
fi
ln -s $1/images/pxeboot/vmlinuz $2/boot/kernel && \
ln -s $1/images/pxeboot/initrd.img $2/boot/initramfs/distribution && \
mkdir -p $2/boot/efi/boot/ && \
ln -s $1/images/pxeboot/rootfs.img $2/ && \
(mcopy -i $1/images/efiboot.img ::efi/redhat/grubx64.efi $2/boot/efi/boot/ || \
mcopy -i $1/images/efiboot.img ::efi/boot/grubx64.efi $2/boot/efi/boot/) && \
mcopy -i $1/images/efiboot.img ::efi/boot/bootx64.efi $2/boot/efi/boot/
@@ -1,5 +0,0 @@
label: RedHat CoreOS %%VERSION%% %%ARCH%% (Default Profile)
kernelargs: quiet random.trust_cpu=on ignition.firstboot ignition.platform.id=metal
#the above boots and will have ignition agent phone home and await instructions
#kernelargs: quiet random.trust_cpu=on ignition.firstboot ignition.platform.id=metal coreos.inst=yes coreos.inst.install_dev=sda
#the above runs the installer to write to disk
@@ -1,97 +0,0 @@
reverse_uuid() {
echo $1 | head -c 8 | tail -c 2
echo $1 | head -c 6 | tail -c 2
echo $1 | head -c 4 | tail -c 2
echo $1 | head -c 2 | tail -c 2
echo $1 | head -c 13 | tail -c 2
echo $1 | head -c 11 | tail -c 2
echo $1 | head -c 18 | tail -c 2
echo $1 | head -c 16 | tail -c 2
echo $1 | tail -c 18 | sed -e s/-//
}
setdebopt() {
debconf-set $1 $2
echo d-i $1 $3 $2 >> /preseed.cfg
}
dhuuid=$(reverse_uuid $(cat /sys/devices/virtual/dmi/id/product_uuid))
dhcpid=$(mktemp)
mkdir -p /etc/confluent
cp /tls/* /etc/ssl/certs/
for nic in $(ip link | grep mtu|grep -v LOOPBACK|cut -d: -f 2|sed -e 's/ //'); do
ip link set $nic up
done
for nic in $(ip link | grep mtu|grep -v LOOPBACK|grep LOWER_UP|cut -d: -f 2|sed -e 's/ //'); do
if udhcpc -i $nic -p $dhcpid -t 2 -T 2 -n -x 93:0007 -x 97:00$dhuuid -q; then
/opt/confluent/bin/copernicus > /etc/confluent/confluent.info
if grep ^MANAGER:.*\\. /etc/confluent/confluent.info ; then
break
fi
fi
ip -4 flush dev $nic
done
mgr=$(grep ^MANAGER:.*\\. /etc/confluent/confluent.info|head -n 1|cut -d: -f 2|sed -e 's/ //')
nodename=$(grep ^NODENAME: /etc/confluent/confluent.info|head -n 1|cut -d: -f 2|sed -e 's/ //')
/opt/confluent/bin/clortho $nodename $mgr > /etc/confluent/confluent.apikey
apikey=$(cat /etc/confluent/confluent.apikey)
cd /etc/confluent
wget --header="CONFLUENT_NODENAME: $nodename" --header="CONFLUENT_APIKEY: $apikey" https://$mgr/confluent-api/self/deploycfg
cd -
predir=$(mktemp -d)
cd $predir
profile=$(grep ^profile: /etc/confluent/deploycfg|cut -d ' ' -f 2)
wget https://$mgr/confluent-public/os/$profile/scripts/pre.sh
chmod u+x pre.sh
wget https://$mgr/confluent-public/os/$profile/preseed.cfg
mv preseed.cfg /
setdebopt auto-install/enable true boolean
setdebopt partman/early_command $predir/pre.sh string
cd -
ip -4 a flush dev $nic
setdebopt netcfg/choose_interface $nic select
setdebopt netcfg/get_hostname $nodename string
v4cfgmeth=$(grep ipv4_method: /etc/confluent/deploycfg |cut -d: -f 2|sed -e 's/ //')
if [ "$v4cfgmeth" = "static" ]; then
setdebopt netcfg/disable_dhcp true boolean
v4addr=$(grep ^ipv4_address: /etc/confluent/deploycfg|cut -d: -f 2|sed -e 's/ //')
v4gw=$(grep ^ipv4_gateway: /etc/confluent/deploycfg|cut -d: -f 2| sed -e 's/ //')
if [ "$v4gw" = "null" ]; then
v4gw=""
fi
v4nm=$(grep ^ipv4_netmask: /etc/confluent/deploycfg|cut -d: -f 2|sed -e 's/ //')
setdebopt netcfg/get_netmask $v4nm string
setdebopt netcfg/get_ipaddress $v4addr string
setdebopt netcfg/confirm_static true boolean
if [ ! -z "$v4gw" ]; then
setdebopt netcfg/get_gateway $v4gw string
fi
namesrvs=$(sed -n '/^nameservers:/,/^[^-]/p' /etc/confluent/deploycfg|grep ^- | cut -d ' ' -f 2|sed -e 's/ //')
for namesrv in "$namesrvs"; do
setdebopt netcfg/get_nameservers $namesrv string
done
elif [ "$v4cfgmeth" = "dhcp" ]; then
setdebopt netcfg/disable_dhcp false boolean
setdebopt netcfg/confirm_static false boolean
fi
rootpass=$(grep ^rootpassword: /etc/confluent/deploycfg|cut -d ' ' -f 2|sed -e 's/ //')
if [ "$rootpass" = null ] || [ -z "$rootpass" ]; then
setdebopt passwd/root-login false boolean
else
setdebopt passwd/root-login true boolean
setdebopt passwd/root-password-crypted $rootpass string
fi
setdebopt time/zone $(grep ^timezone: /etc/confluent/deploycfg|cut -d ' ' -f 2|sed -e 's/ //') string
ntpsrvs=$(sed -n '/^ntpservers:/,/^[^-]/p' /etc/confluent/deploycfg|grep ^- | cut -d ' ' -f 2|sed -e 's/ //')
for ntpsrv in "$ntpsrvs"; do
setdebopt clock-setup/ntp true boolean
setdebopt clock-setup/ntep-server $ntpsrv string
done
#setdebopt console-setup/layoutcode $(grep ^keymap: /etc/confluent/deploycfg|cut -d ' ' -f 2) string
setdebopt debian-installer/locale $(grep ^locale: /etc/confluent/deploycfg|cut -d ' ' -f 2) select
domainname=$(grep ^dnsdomain: /etc/confluent/deploycfg|cut -d ' ' -f 2)
if [ ! -z "$domainname" ] && [ "$domainname" != "null" ]; then
setdebopt netcfg/get_domain $domainname string
fi

Some files were not shown because too many files have changed in this diff Show More