All Questions
3,707
questions
-1
votes
1
answer
37
views
How do I perform a smear between two dataframes in python/pandas? [duplicate]
I have two dataframes and I need to perform a smear (if that is what it's generally called). Basically the first one is smaller (5 million rows) and the other is 40 million rows. I want to add the ...
- 1
0
votes
1
answer
32
views
Why `pd.merge()` still works even though the `on` column is located in the index
Consider the following example:
import pandas as pd
df1 = pd.DataFrame(
data={'A': range(5)},
index=range(10, 15),
).rename_axis(index='ID')
df2 = df1.add(100).reset_index()
print(df1)
# ...
- 319
0
votes
0
answers
42
views
Speed up for loop to merge two dataframes in pandas
This is my code:
import time
st = time.time()
# Determine the smaller DataFrame
if len(purp_expanded) <= len(demog_expanded):
smaller_df = purp_expanded
larger_df = demog_expanded
...
- 665
0
votes
0
answers
22
views
Python Pandas: Select matching value from other table with comparison into each row without matching key [duplicate]
I have a structure of events happening for entities in table A.
Each event is linked to a process in table B.
Unfortunately I don't have the process ID in table A only a timestamp. Table B includes ...
-1
votes
0
answers
87
views
Pandas : Merge Two Dataframe using Condition
I Have a question about how to merge dataframe using conditon.
I have two dataframe
df1
Pro fg material_df1 qty_df1
1 f01 m01 1
1 f01 m02 1
1 f01 m03 1
1 f01 m05 1
2 f01 m01 2
2 ...
- 19
0
votes
1
answer
33
views
I want to lookup values from pivot, if value does not found then use N using Pandas Dataframe [duplicate]
Input df
This is pivot I have.I want to lookup only zero value if there in above pivot and print N if value absent.
Expected df
In excel i am able to do comparison in these 2 df with sumif function
...
- 15
2
votes
2
answers
38
views
Merge Dataframe based on substring column labeld while keep the original columns label
I have a dataframe having columns with the a label pattern (name/startDateTime/endDateTime)
import pandas as pd
pd.DataFrame({
"[RATE] BOJ presser/2024-03-19T07:30:00Z/2024-03-19T10:30:00Z&...
- 213
0
votes
2
answers
58
views
fill in null values of one pandas dataframe with another dataframe [duplicate]
I would like to fill in null values of one pandas dataframe with another dataframe (and multiple times, with multiple dataframes).
Example:
df_A
A B C
index
0 3 5....
- 33
0
votes
0
answers
36
views
Fuzzy Match 2 Large Pandas Dataframes
I have 2 pandas dataframes that both contain company names. I want to left join df1(~10k rows) with df2(~1.6m rows) on company names using a fuzzy match. My current function takes too long to run, so ...
- 27
1
vote
1
answer
55
views
Join 2 dataframes with same column but different values (Python) [duplicate]
I want to join 2 tables, with same column names, table 1 has data universe with some NA's, table 2 has valid data for which table 1 has NA values.
I want to populate NA value in table 1 based on table ...
1
vote
2
answers
39
views
Merge on one column if condition met else merge on another column
So consider these dfs:
data1 = [{'name': 'Muhammad', 'age_x': 20, 'city_x': 'Karachi', 'number' : 1001},
{'name': 'Ali', 'age_x': 19, 'city_x': 'Lahore', 'number' : 1002},
{'name': '...
- 171
1
vote
1
answer
49
views
Python Pandas: Join on case-insensitive and stripped key
Left Dataframe MASTER =
Fname
Lname
Amount
John
Smith-Richards
David
O'Brien
Right Dataframe ORDERS =
Fname
Lname
Amount
DAVID
OBRIEN
36
john
smith richards
11
Expected output Dataframe =
...
- 13
2
votes
4
answers
68
views
Avoiding Merge In Pandas
I have a data frame that looks like this :
I want to group the data frame by #PROD and #CURRENCY and replace TP with the contents of the Offshore data in the Loc column Without creating two data ...
- 894
0
votes
1
answer
35
views
Merging two Pandas dataframes without a primary keys but using latest dates instead [duplicate]
I have two pandas dataframes that looks like:
df1 records the students and their mock exam score and the mock exam date:
ID Mock_Date Student_ID Mock_score
1 14/3/...
- 445
2
votes
2
answers
58
views
join/merge multiple pandas dataframes with blending of values on one column
I have two pandas dataframes width unique column names except:
year
student_id
student_name
I would like to merge on these 3 columns (year, student_id, student_name), however sometimes the ...
- 33