Python Module and parser library issues

Hey everyone,

Still very new with Python (all versions).
I installed a brand new kali release 2020.4.
But I am having problems with some python scripts (a CVE script that I got).

When I run it I get the following error:
soup = BeautifulSoup(html_content,features="lxml") File "/usr/share/offsec-awae-wheels/beautifulsoup4-4.9.1-py2-none-any.whl/bs4/__init__.py", line 245, in __init__ bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: lxml. Do you need to install a parser library?

looking up the error the solutions seems to suggest to install lxml by running: sudo pip3 install lxml or sudo pip install lxml or sudo apt-get install libxml2-dev libxslt-dev python-dev
running the pip installs returns the following message: Requirement already satisfied: lxml in /usr/lib/python3/dist-packages (4.6.2)

So I tried to do the following: export PYTHONPATH=/usr/lib/python3/dist-packages rerun my script in the same terminal but still I get the error: Couldn't find a tree builder with the features you requested: lxml. Do you need to install a parser library?

So to say; I am lost and new to this python scripting (been new for awhile :P)

Would be grateful for a helping hand\eye to fix this up for me (its a fresh install of the distro too)

Always happy to help others. 100% human

https://www.mindfueldaily.com/livewell/thank-you/

Comments

  • edited February 9

    Looking at the error here:

    soup = BeautifulSoup(html_content,features="lxml") File "/usr/share/offsec-awae-wheels/beautifulsoup4-4.9.1-py2-none-any.whl/bs4
    

    especially the 4-4.9.1-py2 would IMO make me think that the script is using python2 and beautifulsoup for python version 2.

    So this is what I would recommend doing:

    Install pip2 (https://pip.pypa.io/en/stable/installing/)

    and then do pip2 (or just pip) install lxml.

    Let me know if it helps!

    PS: you might need to also upgraded setuptools on pip2, you can do this with pip2 install --upgrade setuptools

    Edit: another PS (lol!) I'd honestly recommend just learning python3, not that python2 is much different, just a few syntax differences, but it's just that it's EOL now and isn't supported anymore.

  • Thanks @HadesSec
    yeah no luck in even installing pip2
    curled the file and ran it and got the following error:
    Traceback (most recent call last): File "get-pip.py", line 24244, in <module> main() File "get-pip.py", line 199, in main bootstrap(tmpdir=tmpdir) File "get-pip.py", line 82, in bootstrap from pip._internal.cli.main import main as pip_entry_point File "/tmp/tmp2JmL1p/pip.zip/pip/_internal/cli/main.py", line 60 sys.stderr.write(f"ERROR: {exc}")

    But a good nose you had for pointing out the python2 (py2) ...
    I guess its python3 now for me all the way :)

    Wonder if there is a way to convert the script ... (google )

    Always happy to help others. 100% human

    https://www.mindfueldaily.com/livewell/thank-you/

  • Type your comment> @acidbat said:

    Thanks @HadesSec
    yeah no luck in even installing pip2
    curled the file and ran it and got the following error:
    Traceback (most recent call last): File "get-pip.py", line 24244, in <module> main() File "get-pip.py", line 199, in main bootstrap(tmpdir=tmpdir) File "get-pip.py", line 82, in bootstrap from pip._internal.cli.main import main as pip_entry_point File "/tmp/tmp2JmL1p/pip.zip/pip/_internal/cli/main.py", line 60 sys.stderr.write(f"ERROR: {exc}")

    But a good nose you had for pointing out the python2 (py2) ...
    I guess its python3 now for me all the way :)

    Wonder if there is a way to convert the script ... (google )

    Did you run get-pip.py in python2 or 3? Honestly, I've absolutely NEVER had any issues with that python script.

  • lol whoops I'm an idiot - installed it via python3 successfully.
    Successfully installed pip-21.0.1
    Thanks mate - will see how this goes

    Always happy to help others. 100% human

    https://www.mindfueldaily.com/livewell/thank-you/

  • Type your comment> @acidbat said:

    lol whoops I'm an idiot - installed it via python3 successfully.
    Successfully installed pip-21.0.1
    Thanks mate - will see how this goes

    Happy to see it work! Was a bit nervous and stumped for a second haha.
    For sure! Let me know how it goes!

  • edited February 9

    @acidbat
    in your error message File "/usr/share/offsec-awae-wheels/beautifulsoup4-4.9.1-py2-none-any.whl/bs4/__init__.py", especially the beautifulsoup4-4.9.1-py2-none-any.whl part, means:
    this package is part of beautifulsoup version 4, 4.9.1 to be specific, but it is THE ONE THAT IS SUITABLE FOR PYTHON2, not picky about your OS, nor your processor architecture

    partially disagree with @HadesSec, i don't think this -py2 means the exploit script is written in python2, but i do think it means this beautifulsoup that comes with offsec-awae install, is only made for use with python2
    edit: i see what exploit are you dealing with now haha! yeah the exploit script is in python3

    then i guess you should install beautifulsoup4-4.9.1-py2-none-any.whl's brother i.e. beautifulsoup4-4.9.1-py3-none-any.whl? link here

    just realize inline not obvious is largely due to my display setting, sorry for the spammy tries, also


    Note: TazWake is not going to be available much in August.
    gcc -Wall -Wextra -g
    

    what_what

  • edited February 9

    <nonsense removed, still no idea based on what word is a lib located>

    just realize inline not obvious is largely due to my display setting, sorry for the spammy tries, also


    Note: TazWake is not going to be available much in August.
    gcc -Wall -Wextra -g
    

    what_what

  • Type your comment> @bitn4b said:

    @acidbat
    in your error message File "/usr/share/offsec-awae-wheels/beautifulsoup4-4.9.1-py2-none-any.whl/bs4/__init__.py", especially the beautifulsoup4-4.9.1-py2-none-any.whl part, means:
    this package is part of beautifulsoup version 4, 4.9.1 to be specific, but it is THE ONE THAT IS SUITABLE FOR PYTHON2, not picky about your OS, nor your processor architecture

    Ok, that is good to know and thank you for explaining.

    partially disagree with @HadesSec, i don't think this -py2 means the exploit script is written in python2, but i do think it means this beautifulsoup that comes with offsec-awae install, is only made for use with python2
    edit: i see what exploit are you dealing with now haha! yeah the exploit script is in python3

    I admire people who can just read code and understand what needs to be done right away. Seriously think I course in python3 would do me some good.

    then i guess you should install beautifulsoup4-4.9.1-py2-none-any.whl's brother i.e. beautifulsoup4-4.9.1-py3-none-any.whl? link here

    Awesome, thank you - will look at this at home (where the problematic distro lives)
    Any tips on how to install *.whl files?

    Always happy to help others. 100% human

    https://www.mindfueldaily.com/livewell/thank-you/

  • Type your comment> @acidbat said:

    Awesome, thank you - will look at this at home (where the problematic distro lives)
    Any tips on how to install *.whl files?

    Apparently this is the way to do it: pip install FILE.whl

    Always happy to help others. 100% human

    https://www.mindfueldaily.com/livewell/thank-you/

  • edited February 10

    @acidbat said:
    I admire people who can just read code and understand what needs to be done right away. Seriously think I course in python3 would do me some good.

    uh man i just see command print come with parentheses and assume it's python3, (should be right...i guess? have you fixed your distro?)
    i unfortunately don't know shit about code >_> every time i try to learn i just get overwhelmed by so many new words >_>

    but seriously have you fixed your python...thing?

    just realize inline not obvious is largely due to my display setting, sorry for the spammy tries, also


    Note: TazWake is not going to be available much in August.
    gcc -Wall -Wextra -g
    

    what_what

  • No dice yet - I didn't get too much time last night.
    installing the .whl files did not work (said it was already installed and the python2 one did not meet my requirements of distro '''ERROR: beautifulsoup4-4.9.3-py2-none-any.whl is not a supported wheel on this platform.'''
    will download beautifulsoup4-4.9.1-py2-none-any.whl and see what happens - expecting it will be the same.

    Also tried a convert from python2 to python3 script - got little further but still some modules complaints.
    whats this space

    Always happy to help others. 100% human

    https://www.mindfueldaily.com/livewell/thank-you/

  • bro you ALREADY HAVE beautifulsoup4-4.9.1-py2-none-any.whl, it is the reason why your exploit script errored out in the first place aaaahhhhhhhhhhhhh
    in my original answer i think you need to install beautifulsoup4-4.9.1-py3-none-any.whl, the -py3 one! the -py3 one!

    bro i think you need to try pip3 install beautifulsoup4-4.9.1-py3-none-any.whl
    specify! 3! wherever! you! can!
    including when you run your exploit python3 <exploit>

    (pls work)

    just realize inline not obvious is largely due to my display setting, sorry for the spammy tries, also


    Note: TazWake is not going to be available much in August.
    gcc -Wall -Wextra -g
    

    what_what

  • edited February 10

    <i really dont know anymore>

    just realize inline not obvious is largely due to my display setting, sorry for the spammy tries, also


    Note: TazWake is not going to be available much in August.
    gcc -Wall -Wextra -g
    

    what_what

  • Type your comment> @bitn4b said:

    bro you ALREADY HAVE beautifulsoup4-4.9.1-py2-none-any.whl, it is the reason why your exploit script errored out in the first place aaaahhhhhhhhhhhhh
    in my original answer i think you need to install beautifulsoup4-4.9.1-py3-none-any.whl, the -py3 one! the -py3 one!

    bro i think you need to try pip3 install beautifulsoup4-4.9.1-py3-none-any.whl
    specify! 3! wherever! you! can!
    including when you run your exploit python3 <exploit>

    (pls work)

    Yeah did that - installed it (yay)

    python3 exploit.py

    Traceback (most recent call last):
      File "exploit.py", line 16, in <module>
        from random_words import RandomWords
    ModuleNotFoundError: No module named 'random_words'
    

    Module is installed

    sudo pip3 install Random-Word                                                                                                                                                                                                        1 ⨯
    Requirement already satisfied: Random-Word in /usr/local/lib/python3.9/dist-packages (1.0.6)
    Requirement already satisfied: nose in /usr/local/lib/python3.9/dist-packages (from Random-Word) (1.3.7)
    Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from Random-Word) (2.25.1)
    

    I feel like season2 of LOST

    Always happy to help others. 100% human

    https://www.mindfueldaily.com/livewell/thank-you/

  • edited February 10

    i know it now!
    it should be from random_word import RandomWords!!!!
    there's no s after random_word! the original script made a typo!

    just realize inline not obvious is largely due to my display setting, sorry for the spammy tries, also


    Note: TazWake is not going to be available much in August.
    gcc -Wall -Wextra -g
    

    what_what

  • Type your comment> @bitn4b said:

    then edit the script import part to be from Random-Word import RandomWords?

    i dont feel a thing anymore

    SFA mate.

    Thank you for your help so far - I'll park this crazy project for now. I did root the machine on my other VM but would be nice to align it all to work properly.
    And they say python is the easiest one to learn --- yeah okay -

    Always happy to help others. 100% human

    https://www.mindfueldaily.com/livewell/thank-you/

  • @acidbat
    no no no pls try edit the script import part to be from random_word import RandomWords!
    it is underscore (my bad) but without s followed pls try it ahhhh i tried it and it worked!

    just realize inline not obvious is largely due to my display setting, sorry for the spammy tries, also


    Note: TazWake is not going to be available much in August.
    gcc -Wall -Wextra -g
    

    what_what

  • edited February 10

    @acidbat
    ahhh don't park it man we are so close! it is not crazy it is the original writer made a typo! typo is not crazy it's careless and stupid (or intended) it's not your fault!

    pls just try edit the script import part to be from random_word import RandomWords it just take 3 seconds and if it still wont work we will then park it!

    just realize inline not obvious is largely due to my display setting, sorry for the spammy tries, also


    Note: TazWake is not going to be available much in August.
    gcc -Wall -Wextra -g
    

    what_what

  • see the original exploit said from random_words but it should be from random_word, no s, and we're done

    just realize inline not obvious is largely due to my display setting, sorry for the spammy tries, also


    Note: TazWake is not going to be available much in August.
    gcc -Wall -Wextra -g
    

    what_what

  • @bitn4b I appreciate your passion on this :D.

    So I got further (thank you)

    But

    line 92, in <module>
        project_name = r.random_word()
    AttributeError: 'RandomWords' object has no attribute 'random_word'
    

    Always happy to help others. 100% human

    https://www.mindfueldaily.com/livewell/thank-you/

  • Shout out to @bitn4b
    Script is executing (finally) - python2 to python3 issues.
    Mate you are a legend for sticking with this :D - so thank you once again.

    Having some environmental issue - but I don't think that is the script

    Always happy to help others. 100% human

    https://www.mindfueldaily.com/livewell/thank-you/

  • i need good book for learning python zero to hero

Sign In to comment.