All Questions
Tagged with pandas beautifulsoup
1,419
questions
-2
votes
1
answer
49
views
trying to find out the logic of this page: approx ++ 100 results stored - and parsed with Python & BS4
trying to find out the logic that is behind this page:
we have stored some results in the following db:
https://www.raiffeisen.ch/rch/de/ueber-uns/raiffeisen-gruppe/organisation/raiffeisenbanken/...
- 1,221
0
votes
1
answer
30
views
trying to apply a bs4-approach to wikipedia-page: results do not store in a df
due to the fact that scraping on Wikipedia is a very very common technique - where we can use an appropiate approach to work with many many different jobs - i did have some issues with getting back ...
- 1,221
2
votes
2
answers
49
views
Convert string to dataframe after extracting using BeautifulSoup
import requests
import pandas as pd
from bs4 import BeautifulSoup as bs
from io import StringIO
url = "https://www.tickertape.in/stocks/oil-and-natural-gas-corporation-ONGC"
r = requests....
-1
votes
1
answer
47
views
Scraping the first table from a website using BeautifulSoup
I am trying to scrape the first table which is the ten countries with biggest biggest market capitalization table I have written the code but the table is not printed it is giving me out that is not ...
- 25
-4
votes
1
answer
100
views
Issue creating CSV from webscraping [closed]
I want to scrape this website https://www.thesoldiersproject.org/which-exo-members-are-in-the-military/ to retrieve the member name, enlisted date, and discharge date. But after I wrote and run my ...
- 5
0
votes
1
answer
34
views
Inserting DOM element to a HTML changing charecter "<" to html/xml character "<" in Python using Pandas
I want to edit a html file and make a column editable in a table. I am using pandas and BeautifulSoup in python.
Code Snippet:
import pandas as pd
from bs4 import BeautifulSoup
with open("../...
- 29
1
vote
2
answers
72
views
How to parse out text from PDF into pandas dataframe
I am working on scraping data from several infographics on ridership data for Amtrak. I want to collect the yearly ridership #s and addresses of each station in the US.
Here is my code for one ...
- 879
0
votes
1
answer
50
views
Unable to create similar column headers using list comprehension as pandas does for a particular table
I'm trying to scrape headers of a table from a webpage using list comprehension. The problem I'm facing is that when I create the same headers using pandas, the appearance is vastly different. Just to ...
- 170
2
votes
2
answers
94
views
How to scrape links from summary section / link list of wikipedia?
update: many thanks for the replies - the help and all the efforts! some additional notes i have added. below (at the end)
howdy i am trying to scrape all the Links of a large wikpedia page from the &...
- 1,221
0
votes
1
answer
66
views
How to automate scraping wikipedia-info box specifically and print the data using python for more (other) wiki page?
How to automate scraping wikipedia info box specifically and print the data using python for any wiki page? My task is to automate printing the wikipedia infobox data. And that said i found out that ...
- 1,221
-1
votes
2
answers
47
views
Why does Pandas not scrape the second table?
I want to scrape the 2 tables, but only get the result of the first table.
Why? I'm using the same logic for both tables.
import requests
from bs4 import BeautifulSoup
import pandas as pd
# URL to ...
1
vote
1
answer
50
views
Why pandas read_html automatically remove decimal separator?
I've been trying to scrape a table from a website, but for some reason pandas automatically turns every column into a string and therefore some values become totally useless.
For example, 0,62 becomes ...
- 13
1
vote
1
answer
53
views
How to extract table from webpage that requires click/toggle?
I'm trying to extract tables from this webpage, but am only able to get the pitching table for example. I want to get the hitting table as well, which would in theory be this URL:
https://www.covers....
- 39
0
votes
2
answers
53
views
How to convert scraped HTML document to a dataframe?
I am trying to scrape football players' data from the website FBRef, I got the data from the website as a bs4.element.ResultSet object.
Code:
import requests
from bs4 import BeautifulSoup
import ...
- 19
2
votes
1
answer
293
views
Scraping MLB daily lineups from rotowire using python
I am trying to scrape the MLB daily lineup information from here: https://www.rotowire.com/baseball/daily-lineups.php
I am trying to use python with requests, BeautifulSoup and pandas.
My ultimate ...
- 71