Files

Derek Feichtinger 6da52b2950 Version for python 3.6

- basic syntax fixes
- bytestring to utf8 conversions
- __init__.py import problem fix

2019-05-11 12:34:44 +02:00

17 KiB

Raw Permalink Blame History

todo for ldapuserdir

Bugs and feature requests

Bugs and feature requests

RESOLVED Error when displayName not set

CLOSED: [2016-03-10 Thu 15:44]

State "RESOLVED" from "BUG" [2016-03-10 Thu 15:44]
State "BUG" from [2016-03-09 Wed 14:13]

dfeich@dflt1w:~/Documents/merlin-accounting$ ldapuserdir-ctl -n -u '*sz*'
monoszlai_b:36001:IGNORE:710:Monoszlai Balazs:/afs/psi.ch/user/m/monoszlai_b:/bin/bash
kupiszewski_p:4822:IGNORE:520:Kupiszewski Piotr:/afs/psi.ch/user/k/kupiszewski_p:/bin/bash
szijarto_r:3107:IGNORE:420:Szijarto Rita:/afs/psi.ch/user/s/szijarto_r:/bin/tcsh
janasz_f:4359:IGNORE:420:Janasz Filip:/afs/psi.ch/user/j/janasz_f:/bin/bash
Traceback (most recent call last):
File "/opt/anaconda/python2.7/bin/ldapuserdir-ctl", line 4, in <module>
 __import__('pkg_resources').run_script('ldapuserdir==2.1.2', 'ldapuserdir-ctl')
File "/opt/anaconda/python2.7/lib/python2.7/site-packages/setuptools-20.1.1-py2.7.egg/pkg_resources/__init__.py", line 724, in run_script
 
File "/opt/anaconda/python2.7/lib/python2.7/site-packages/setuptools-20.1.1-py2.7.egg/pkg_resources/__init__.py", line 1657, in run_script
 
File "/opt/anaconda/python2.7/lib/python2.7/site-packages/ldapuserdir-2.1.2-py2.7.egg/EGG-INFO/scripts/ldapuserdir-ctl", line 345, in <module>
 
File "build/bdist.linux-x86_64/egg/ldapuserdir/ldapuserdir.py", line 359, in list_users_etcpwd
KeyError: 'displayName'

RESOLVED Handling of ldap server returning slices (paging)

CLOSED: [2016-03-03 Thu 15:07]

State "RESOLVED" from "BUG" [2016-03-03 Thu 15:07]
State "BUG" from [2016-02-16 Tue 16:03]

In the case of big groups, ldap returns the results with the attribute name member being extended for the slice information. Need to implement the client to be able to deal with this.
- Dani's docu may help https://intranet.psi.ch/AIT/AdLdapClientConfiguration
Kai Kaminski supplied a fix in #c0fa2b0bbb7bfd58f4f5197936d1ca487fdb8c4d

WORKAROUND group list returning some elements with dn=None

CLOSED: [2016-02-16 Tue 17:27]

State "WORKAROUND" from "BUG" [2016-02-16 Tue 17:27]
State "BUG" from [2016-02-16 Tue 16:03]

Invoking ldapuserdir-ctl in the following way leads to error

ldapuserdir-ctl --debug --group-ou='dc=d,dc=psi,dc=ch' 'p1299*'

  p12990:IGNORE:12990:
  p12991:IGNORE:12991:
  p12992:IGNORE:12992:
  p12993:IGNORE:12993:
  p12994:IGNORE:12994:
  p12995:IGNORE:12995:
  p12996:IGNORE:12996:
  p12997:IGNORE:12997:
  p12998:IGNORE:12998:
  p12999:IGNORE:12999:
  Traceback (most recent call last):
    File "/home/dfeich/conda-envs/build-test/bin/ldapuserdir-ctl", line 6, in <module>
      exec(compile(open(__file__).read(), __file__, 'exec'))
    File "/home/dfeich/projects/ldapuserdir/bin/ldapuserdir-ctl", line 341, in <module>
      verbose=flag_verbose)
    File "/home/dfeich/projects/ldapuserdir/ldapuserdir/ldapuserdir.py", line 537, in list_groups
      sys.stdout.write("%s:IGNORE:%s:" % (entry['cn'][0], gid))
  TypeError: list indices must be integers, not str

Reason: The search in get_groups_struct

 r = self.search_s_reconn(group_ou, ldap.SCOPE_SUBTREE, srch)

returns a result list which contains (dn, dict) pairs with the exception of the last 2 entries which are (None, list).

  [ ...
  ('CN=p12999,OU=Groups,OU=Experiment,OU=IT,DC=d,DC=psi,DC=ch',
    {'cn': ['p12999'],
     'dSCorePropagationData': ['16010101000000.0Z'],
     'distinguishedName': ['CN=p12999,OU=Groups,OU=Experiment,OU=IT,DC=d,DC=psi,DC=ch'],
     'gidNumber': ['12999'],
     'groupType': ['-2147483646'],
     'instanceType': ['4'],
     'msSFU30GidNumber': ['12999'],
     'name': ['p12999'],
     'objectCategory': ['CN=Group,CN=Schema,CN=Configuration,DC=d,DC=psi,DC=ch'],
     'objectClass': ['top', 'group'],
     'objectGUID': ['jx\xd9\x7f/\x16\xebJ\xb7\x1c\xf3~\n\xd4\x17\x95'],
     'objectSid': ['\x01\x05\x00\x00\x00\x00\x00\x05\x15\x00\x00\x00x.\x9d\x13\x931\xda\x1cC\x17\n2\xa9\x1a\x01\x00'],
     'sAMAccountName': ['p12999'],
     'sAMAccountType': ['268435456'],
     'uSNChanged': ['817289847'],
     'uSNCreated': ['817289847'],
     'whenChanged': ['20160215133759.0Z'],
     'whenCreated': ['20160215133757.0Z']}),
   (None, ['ldaps://d.psi.ch/CN=Configuration,DC=d,DC=psi,DC=ch']),
   (None, ['ldaps://d.psi.ch/CN=Schema,CN=Configuration,DC=d,DC=psi,DC=ch'])]

Not clear how these entries come to pass. It does not happen if I use the additional classification 'ou=IT'.

ldapuserdir-ctl --debug --group-ou='ou=IT,dc=d,dc=psi,dc=ch' 'p1299*'

The corresponding ldapsearch yields no clue abou these additional elements. Probably it filters them out. Discussing with Björn and looking it up in AD, we can see that these CN paths exist for the config information.

  ldapsearch -x -w 'TBVsK5zOfqMyxVmXco7y' \
             -D 'CN=linux_ldap,OU=Services,OU=IT,DC=d,DC=psi,DC=ch' \
             -H "ldaps://d.psi.ch:636" -b "dc=d,dc=PSI,dc=ch" \
             '(&(objectClass=Group)(msSFU30GidNumber=*)(cn=p1299*))'

# extended LDIF
#
# LDAPv3
# base <dc=d,dc=PSI,dc=ch> with scope subtree
# filter: (&(objectClass=Group)(msSFU30GidNumber=*)(cn=p1299*))
# requesting: ALL
#

# p12990, Groups, Experiment, IT, d.psi.ch
dn: CN=p12990,OU=Groups,OU=Experiment,OU=IT,DC=d,DC=psi,DC=ch
objectClass: top
objectClass: group
cn: p12990
distinguishedName: CN=p12990,OU=Groups,OU=Experiment,OU=IT,DC=d,DC=psi,DC=ch
instanceType: 4
whenCreated: 20160215133748.0Z
whenChanged: 20160215133757.0Z
uSNCreated: 817286825
uSNChanged: 817286825
name: p12990
objectGUID:: e0ffV/01TEOscX9xFanuew==
objectSid:: AQUAAAAAAAUVAAAAeC6dE5Mx2hxDFwoy+w4BAA==
sAMAccountName: p12990
sAMAccountType: 268435456
groupType: -2147483646
objectCategory: CN=Group,CN=Schema,CN=Configuration,DC=d,DC=psi,DC=ch
dSCorePropagationData: 16010101000000.0Z
msSFU30GidNumber: 12990
gidNumber: 12990

# p12991, Groups, Experiment, IT, d.psi.ch
dn: CN=p12991,OU=Groups,OU=Experiment,OU=IT,DC=d,DC=psi,DC=ch
objectClass: top
objectClass: group
cn: p12991
distinguishedName: CN=p12991,OU=Groups,OU=Experiment,OU=IT,DC=d,DC=psi,DC=ch
instanceType: 4
whenCreated: 20160215133751.0Z
whenChanged: 20160215133758.0Z
uSNCreated: 817287915
uSNChanged: 817287915
name: p12991
objectGUID:: 9+o2BvtkMkK3pDPbQrTyPw==
objectSid:: AQUAAAAAAAUVAAAAeC6dE5Mx2hxDFwoyLRMBAA==
sAMAccountName: p12991
sAMAccountType: 268435456
groupType: -2147483646
objectCategory: CN=Group,CN=Schema,CN=Configuration,DC=d,DC=psi,DC=ch
dSCorePropagationData: 16010101000000.0Z
msSFU30GidNumber: 12991
gidNumber: 12991

# p12992, Groups, Experiment, IT, d.psi.ch
dn: CN=p12992,OU=Groups,OU=Experiment,OU=IT,DC=d,DC=psi,DC=ch
objectClass: top
objectClass: group
cn: p12992
distinguishedName: CN=p12992,OU=Groups,OU=Experiment,OU=IT,DC=d,DC=psi,DC=ch
instanceType: 4
whenCreated: 20160215133755.0Z
whenChanged: 20160215133758.0Z
uSNCreated: 817289293
uSNChanged: 817289293
name: p12992
objectGUID:: llozR0I690u3dIbXoLFdpg==
objectSid:: AQUAAAAAAAUVAAAAeC6dE5Mx2hxDFwoyhBgBAA==
sAMAccountName: p12992
sAMAccountType: 268435456
groupType: -2147483646
objectCategory: CN=Group,CN=Schema,CN=Configuration,DC=d,DC=psi,DC=ch
dSCorePropagationData: 16010101000000.0Z
msSFU30GidNumber: 12992
gidNumber: 12992

# p12993, Groups, Experiment, IT, d.psi.ch
dn: CN=p12993,OU=Groups,OU=Experiment,OU=IT,DC=d,DC=psi,DC=ch
objectClass: top
objectClass: group
cn: p12993
distinguishedName: CN=p12993,OU=Groups,OU=Experiment,OU=IT,DC=d,DC=psi,DC=ch
instanceType: 4
whenCreated: 20160215133755.0Z
whenChanged: 20160215133758.0Z
uSNCreated: 817289294
uSNChanged: 817289294
name: p12993
objectGUID:: QY8deD1SsUaP+G9ShR3+5A==
objectSid:: AQUAAAAAAAUVAAAAeC6dE5Mx2hxDFwoyhRgBAA==
sAMAccountName: p12993
sAMAccountType: 268435456
groupType: -2147483646
objectCategory: CN=Group,CN=Schema,CN=Configuration,DC=d,DC=psi,DC=ch
dSCorePropagationData: 16010101000000.0Z
msSFU30GidNumber: 12993
gidNumber: 12993

# p12994, Groups, Experiment, IT, d.psi.ch
dn: CN=p12994,OU=Groups,OU=Experiment,OU=IT,DC=d,DC=psi,DC=ch
objectClass: top
objectClass: group
cn: p12994
distinguishedName: CN=p12994,OU=Groups,OU=Experiment,OU=IT,DC=d,DC=psi,DC=ch
instanceType: 4
whenCreated: 20160215133757.0Z
whenChanged: 20160215133759.0Z
uSNCreated: 817289842
uSNChanged: 817289842
name: p12994
objectGUID:: rsAueF9W2kaQ2VW/Qtp1wg==
objectSid:: AQUAAAAAAAUVAAAAeC6dE5Mx2hxDFwoypBoBAA==
sAMAccountName: p12994
sAMAccountType: 268435456
groupType: -2147483646
objectCategory: CN=Group,CN=Schema,CN=Configuration,DC=d,DC=psi,DC=ch
dSCorePropagationData: 16010101000000.0Z
msSFU30GidNumber: 12994
gidNumber: 12994

# p12995, Groups, Experiment, IT, d.psi.ch
dn: CN=p12995,OU=Groups,OU=Experiment,OU=IT,DC=d,DC=psi,DC=ch
objectClass: top
objectClass: group
cn: p12995
distinguishedName: CN=p12995,OU=Groups,OU=Experiment,OU=IT,DC=d,DC=psi,DC=ch
instanceType: 4
whenCreated: 20160215133757.0Z
whenChanged: 20160215133759.0Z
uSNCreated: 817289843
uSNChanged: 817289843
name: p12995
objectGUID:: xEDgBQTscEennB8f7HnrJg==
objectSid:: AQUAAAAAAAUVAAAAeC6dE5Mx2hxDFwoypRoBAA==
sAMAccountName: p12995
sAMAccountType: 268435456
groupType: -2147483646
objectCategory: CN=Group,CN=Schema,CN=Configuration,DC=d,DC=psi,DC=ch
dSCorePropagationData: 16010101000000.0Z
msSFU30GidNumber: 12995
gidNumber: 12995

# p12996, Groups, Experiment, IT, d.psi.ch
dn: CN=p12996,OU=Groups,OU=Experiment,OU=IT,DC=d,DC=psi,DC=ch
objectClass: top
objectClass: group
cn: p12996
distinguishedName: CN=p12996,OU=Groups,OU=Experiment,OU=IT,DC=d,DC=psi,DC=ch
instanceType: 4
whenCreated: 20160215133757.0Z
whenChanged: 20160215133759.0Z
uSNCreated: 817289844
uSNChanged: 817289844
name: p12996
objectGUID:: rzV1wOhoZ0GgI9B9Of+oFA==
objectSid:: AQUAAAAAAAUVAAAAeC6dE5Mx2hxDFwoyphoBAA==
sAMAccountName: p12996
sAMAccountType: 268435456
groupType: -2147483646
objectCategory: CN=Group,CN=Schema,CN=Configuration,DC=d,DC=psi,DC=ch
dSCorePropagationData: 16010101000000.0Z
msSFU30GidNumber: 12996
gidNumber: 12996

# p12997, Groups, Experiment, IT, d.psi.ch
dn: CN=p12997,OU=Groups,OU=Experiment,OU=IT,DC=d,DC=psi,DC=ch
objectClass: top
objectClass: group
cn: p12997
distinguishedName: CN=p12997,OU=Groups,OU=Experiment,OU=IT,DC=d,DC=psi,DC=ch
instanceType: 4
whenCreated: 20160215133757.0Z
whenChanged: 20160215133759.0Z
uSNCreated: 817289845
uSNChanged: 817289845
name: p12997
objectGUID:: 15jnQe7x7kubD0LVnwGhWA==
objectSid:: AQUAAAAAAAUVAAAAeC6dE5Mx2hxDFwoypxoBAA==
sAMAccountName: p12997
sAMAccountType: 268435456
groupType: -2147483646
objectCategory: CN=Group,CN=Schema,CN=Configuration,DC=d,DC=psi,DC=ch
dSCorePropagationData: 16010101000000.0Z
msSFU30GidNumber: 12997
gidNumber: 12997

# p12998, Groups, Experiment, IT, d.psi.ch
dn: CN=p12998,OU=Groups,OU=Experiment,OU=IT,DC=d,DC=psi,DC=ch
objectClass: top
objectClass: group
cn: p12998
distinguishedName: CN=p12998,OU=Groups,OU=Experiment,OU=IT,DC=d,DC=psi,DC=ch
instanceType: 4
whenCreated: 20160215133757.0Z
whenChanged: 20160215133759.0Z
uSNCreated: 817289846
uSNChanged: 817289846
name: p12998
objectGUID:: K0QCSvfKY0+8xBfUx/sp1A==
objectSid:: AQUAAAAAAAUVAAAAeC6dE5Mx2hxDFwoyqBoBAA==
sAMAccountName: p12998
sAMAccountType: 268435456
groupType: -2147483646
objectCategory: CN=Group,CN=Schema,CN=Configuration,DC=d,DC=psi,DC=ch
dSCorePropagationData: 16010101000000.0Z
msSFU30GidNumber: 12998
gidNumber: 12998

# p12999, Groups, Experiment, IT, d.psi.ch
dn: CN=p12999,OU=Groups,OU=Experiment,OU=IT,DC=d,DC=psi,DC=ch
objectClass: top
objectClass: group
cn: p12999
distinguishedName: CN=p12999,OU=Groups,OU=Experiment,OU=IT,DC=d,DC=psi,DC=ch
instanceType: 4
whenCreated: 20160215133757.0Z
whenChanged: 20160215133759.0Z
uSNCreated: 817289847
uSNChanged: 817289847
name: p12999
objectGUID:: anjZfy8W60q3HPN+CtQXlQ==
objectSid:: AQUAAAAAAAUVAAAAeC6dE5Mx2hxDFwoyqRoBAA==
sAMAccountName: p12999
sAMAccountType: 268435456
groupType: -2147483646
objectCategory: CN=Group,CN=Schema,CN=Configuration,DC=d,DC=psi,DC=ch
dSCorePropagationData: 16010101000000.0Z
msSFU30GidNumber: 12999
gidNumber: 12999

# search reference
ref: ldaps://d.psi.ch/CN=Configuration,DC=d,DC=psi,DC=ch

# search result
search: 2
result: 0 Success

# numResponses: 12
# numEntries: 10
# numReferences: 1

I implemented a workaround by filtering out the None elements.

[2019-05-10 Fri] compatibility with python-3.6

RESOLVED simple fixes

CLOSED: [2019-05-11 Sat 09:54]

State "RESOLVED" from "BUG" [2019-05-11 Sat 09:54]
State "BUG" from [2019-05-11 Sat 09:54]

Exceptions: use new syntax
```
except SomeException as err
```
print statements

RESOLVED importer namespace problem

CLOSED: [2019-05-11 Sat 09:55]

State "RESOLVED" from "BUG" [2019-05-11 Sat 09:55]
State "BUG" from [2019-05-11 Sat 09:54]

init.py only works with changing to relative import

from ldapuserdir import LdapUserDir, LdapUserDirError

now must be made explicit with

from ldapuserdir.ldapuserdir import LdapUserDir, LdapUserDirError

RESOLVED hangs in LDAP paging call

CLOSED: [2019-05-11 Sat 12:28]

State "RESOLVED" from "BUG" [2019-05-11 Sat 12:28]
State "BUG" from [2019-05-11 Sat 10:05]

The loop for reading the paged results never reaches the break condition

in ldapuserdir.py:_search_s

  page_ctrl = SimplePagedResultsControl(criticality=True,
                                        size=self.page_size,
                                        cookie='')
  msgid = self._ldap.search_ext(base, scope, filterstr, attrlist,
                                attrsonly,
                                serverctrls=[page_ctrl])

  results = []
  while True:
      _, rdata, _, resp_ctrls = self._ldap.result3(msgid)
      results.extend(rdata)
      self.logger.debug('DEREK: in paging result call: results= %s' % results)
      # .... CUT ....
      # Extract the SimplePagedResultsControl to get the cookie.
      page_ctrls = [c for c in resp_ctrls if c.controlType == SimplePagedResultsControl.controlType]
      if page_ctrls == [] or page_ctrls[0].cookie == '':
          # We're done.
          break
      else:
          # Update the cookie to retrieve the next page.
          page_ctrl.cookie = page_ctrls[0].cookie

The conditions for the break need to be changed. Good resource: https://medium.com/@alpolishchuk/pagination-of-ldap-search-results-with-python-ldap-845de60b90d2

  if not page_ctrls:
      raise RuntimeError("The server ignores RFC 2696 control (paged results)")
  if not page_ctrls[0].cookie:
      # We're done.
      break
  # Update the cookie to retrieve the next page.
  page_ctrl.cookie = page_ctrls[0].cookie

RESOLVED In python3 the ldap calls return bytestrings