[SOLVED] "spi" missing urllib2 - after python 2-3 update

[colebaas]

Like to quickly thank the outstanding work of the Salix crew. Very much appreciated!!
Did a major install/upgrade of Python 2.7 & 3.5 and anaconda.
This apparently broke spi - which is yet an other excellent tool!
After the python upgrade, the following error:

Code: Select all

Traceback (most recent call last):
  File "/usr/libexec/spi.py", line 10, in <module>
	import urllib2
ImportError: No module named urllib2

Tried to [re]install: spi and also ndg_httpsclient and python-mechanize and python-ntlm - all of which point to [ supposedly] including -> urllib2. But no-go.... same spi message about urllib2 missing.
Any help would be much appreciated.

[laprjns]

Did a major install/upgrade of Python 2.7...

The urllib2 lubrary is includeded in the python-2.7.11 package. So what ever you did to upgrade it was most likely responsible. Try reinstalling python, then check to see if urllib2 was installed.

Code: Select all

rich[vars]$ sudo slapt-get --filelist python | grep urllib2
Password: 
/usr/doc/python-2.7.11/Documentation/howto/urllib2.txt
/usr/doc/python-2.7.11/Documentation/library/urllib2.txt
/usr/lib64/python2.7/test/test_urllib2.py
/usr/lib64/python2.7/test/test_urllib2.pyc
/usr/lib64/python2.7/test/test_urllib2.pyo
/usr/lib64/python2.7/test/test_urllib2_localnet.py
/usr/lib64/python2.7/test/test_urllib2_localnet.pyc
/usr/lib64/python2.7/test/test_urllib2_localnet.pyo
/usr/lib64/python2.7/test/test_urllib2net.py
/usr/lib64/python2.7/test/test_urllib2net.pyc
/usr/lib64/python2.7/test/test_urllib2net.pyo
/usr/lib64/python2.7/urllib2.py
/usr/lib64/python2.7/urllib2.pyc
/usr/lib64/python2.7/urllib2.pyo

[colebaas]

Thank you for the quick reply.
So far it appears that changing the environment [in anaconda ] to Python 3.5 breaks spi - removes urllib2.
I am also trying to restore the previous python 2.7 environment to see if that would fix it but that's quite a haul after I've just upgraded 803 packages.

p.s. I used Anaconda3 and pip to incorporate both Python 2.7 & 3.5. It looks like that during the pip run - those packages which anaconda doesn't handle - is where the breakage occurs. Which, again, involves sifting through over 800 packages. [Running a large cluster here] So... work in progress. - Thank you again!

[colebaas]

Here is a rather a elementary question, if you don't mind;
What would be a sensible way to "reinstall python" under Salix, without gutting the entire, installed system?
Incidentally urlib2 still exists under both, python 2.7 & 3.5, even after the upgrade/breakage.

Code: Select all

/usr/lib64/python3.5/test/test_urllib2_localnet.py
/usr/lib64/python3.5/test/test_urllib2.py
/usr/lib64/python3.5/test/__pycache__/test_urllib2_localnet.cpython-35.opt-1.pyc
/usr/lib64/python3.5/test/__pycache__/test_urllib2.cpython-35.opt-2.pyc
/usr/lib64/python3.5/test/__pycache__/test_urllib2_localnet.cpython-35.pyc
/usr/lib64/python3.5/test/__pycache__/test_urllib2net.cpython-35.opt-2.pyc
/usr/lib64/python3.5/test/__pycache__/test_urllib2.cpython-35.opt-1.pyc
/usr/lib64/python3.5/test/__pycache__/test_urllib2net.cpython-35.pyc
/usr/lib64/python3.5/test/__pycache__/test_urllib2.cpython-35.pyc
/usr/lib64/python3.5/test/__pycache__/test_urllib2net.cpython-35.opt-1.pyc
/usr/lib64/python3.5/test/__pycache__/test_urllib2_localnet.cpython-35.opt-2.pyc
/usr/lib64/python3.5/test/test_urllib2net.py
/usr/lib64/python2.7/urllib2.pyc
/usr/lib64/python2.7/urllib2.py
/usr/lib64/python2.7/test/test_urllib2.pyc
/usr/lib64/python2.7/test/test_urllib2_localnet.pyc
/usr/lib64/python2.7/test/test_urllib2_localnet.py
/usr/lib64/python2.7/test/test_urllib2.py
/usr/lib64/python2.7/test/test_urllib2.pyo
/usr/lib64/python2.7/test/test_urllib2net.pyc
/usr/lib64/python2.7/test/test_urllib2net.pyo
/usr/lib64/python2.7/test/test_urllib2_localnet.pyo
/usr/lib64/python2.7/test/test_urllib2net.py
/usr/lib64/python2.7/urllib2.pyo
/usr/doc/python-2.7.11/Documentation/howto/urllib2.txt
/usr/doc/python-2.7.11/Documentation/library/urllib2.txt

[EDIT]
Attempting to change

Code: Select all

import urlib2

to

Code: Select all

import test_urlib2

in "/usr/libexec/spi.py" (as it's shown under python 3.5) did NOT fix spi.
Re-linking [ln -s test_urllib2... urllib2...] in python 3.5 also did NOT seem to do anything for spi.

[laprjns]

What would be a sensible way to "reinstall python" under Salix, without gutting the entire, installed system?

Use slapt-get.

Code: Select all

$ sudo slapt-get --install --reinstall python

[colebaas]

Continued thanks!....
Re-installed Python(2.7), as suggested and still no change --- spi still can Not find urllib2.

Also, grabbed a Salix workstation on which spi was working fine.
"Upgraded/installed" anaconda & python on it, STEP-BY-STEP (one package at a time), to see where/if it breaks spi. This fresh upgrade also DID break spi - as expected.
Comparing the relevant libraries & modules under python 2.7, before and after, however show no difference pre- vs post-upgrade. - same number and size files.
So the problem does not seem to occur in these directories:

Code: Select all

/usr/lib64/python3.5/test/test_urllib2_localnet.py
/usr/lib64/python3.5/test/test_urllib2.py
/usr/lib64/python3.5/test/__pycache__/test_urllib2_localnet.cpython-35.opt-1.pyc
/usr/lib64/python3.5/test/__pycache__/test_urllib2.cpython-35.opt-2.pyc
/usr/lib64/python3.5/test/__pycache__/test_urllib2_localnet.cpython-35.pyc
/usr/lib64/python3.5/test/__pycache__/test_urllib2net.cpython-35.opt-2.pyc
/usr/lib64/python3.5/test/__pycache__/test_urllib2.cpython-35.opt-1.pyc
/usr/lib64/python3.5/test/__pycache__/test_urllib2net.cpython-35.pyc
/usr/lib64/python3.5/test/__pycache__/test_urllib2.cpython-35.pyc
/usr/lib64/python3.5/test/__pycache__/test_urllib2net.cpython-35.opt-1.pyc
/usr/lib64/python3.5/test/__pycache__/test_urllib2_localnet.cpython-35.opt-2.pyc
/usr/lib64/python3.5/test/test_urllib2net.py
/usr/lib64/python2.7/urllib2.pyc
/usr/lib64/python2.7/urllib2.py
/usr/lib64/python2.7/test/test_urllib2.pyc
/usr/lib64/python2.7/test/test_urllib2_localnet.pyc
/usr/lib64/python2.7/test/test_urllib2_localnet.py
/usr/lib64/python2.7/test/test_urllib2.py
/usr/lib64/python2.7/test/test_urllib2.pyo
/usr/lib64/python2.7/test/test_urllib2net.pyc
/usr/lib64/python2.7/test/test_urllib2net.pyo
/usr/lib64/python2.7/test/test_urllib2_localnet.pyo
/usr/lib64/python2.7/test/test_urllib2net.py
/usr/lib64/python2.7/urllib2.pyo
/usr/doc/python-2.7.11/Documentation/howto/urllib2.txt
/usr/doc/python-2.7.11/Documentation/library/urllib2.txt

But where else would spi look for urllib2 ?

p.s.
The actual "break" occurs "somewhere" between installing/upgrading these files:
- OpenBLAS
- OpenCV
- glog
- gflags
- hdf5
- snappy
- lmdb
- leveldb
- ATLAS
This is as far we could narrow it down, so far - none of which has any apparent relation to urllib2. Except may be "snappy".

[gapan]

Did a major install/upgrade of Python 2.7 & 3.5 and anaconda.

What does that mean exactly? What exactly did you do?

You have probably replaced the system python, which is 2.7, with your 3.5. urllib2 does not exist in 3.5. And spi is not the only thing that you broke.

It's most likely that you need to undo everything. I really don't understand why you had to do what you did. You shouldn't touch system libraries if you don't know what you're doing (and want to keep everything running). And you really should learn how to use virtualenv. You can have any number of different python versions and libraries without touching the system libraries at all with it.

If it is not possible to undo everything, then you'll need to reinstall from scratch. And don't touch system libraries again.

[colebaas]

t's most likely that you need to undo everything

@gapan We concur. Which is why we were worried about spi getting clobbered, thinking it may not be the only thing that broke. - Except, strangely enough everything else checks out and is running fine, at least "for the moment".

To "reinstall the whole system, from scratch".... That's funny! - Although potentially sound advice. Running a 1440x server/workstation cluster (Slackware/Salix; probably the largest of its kind) "may" raise some eyebrows. (Not exactly your run-of-the-mill, "I have no sound on my new laptop", kind of problem.)
Nevertheless, we all very much appreciate all the excellent work that you and the entire Salix development team does, and did, over the years. Thank you all, very much!!!

[mimosa]

For future reference, useful tool for keeping track of what was installed (and in what order) is twapake. I don't think it is installed by default. However, obviously, it doesn't work if you go behind the back of package management and break your system.

[colebaas]

Yes, thank you. twapake is very convenient indeed and we have it installed!
- And No, unfortunately it can't perform "magic" - why?? - when one breaks "stuff" outside the scope of built-in pkg. managers.
But hey, one's gotta do what one's gotta do when one has to upgrade ~2100 packages and install 800 new ones, most of which are still in alpha.
But hey, we were still unable to break Slack and we obviously "tried" real hard, this time around!
[spi is still dead but will do an autopsy provided enough time.]