Categories
Mastering Development

ValueError: You are trying to merge on object and int64 columns when use pandas merge

The test.csv data likes this: device_id,upload_time,latitude,longitude,mileage,other_vals,speed,upload_time_1 11115304371,2020-08-05 05:10:05+00:00,23.140366,114.18685,0,,0,202008 1234,2020-08-05 05:10:33+00:00,22.994716,114.2998,0,,0,202008 11115304371,2020-08-05 05:20:55+00:00,22.994716,114.2998,0,,3.8,202008 11115304371,2020-08-05 05:24:02+00:00,22.994916,114.299683,0,,2.1,202008 11115304371,2020-08-05 05:24:30+00:00,22.99545,114.2998,0,,6.5,202008 11115304371,2020-08-05 05:29:30+00:00,22.995433,114.299766,0,,3.4,202008 11115304371,2020-08-05 05:34:30+00:00,22.995433,114.299766,0,,3.4,202008 11115304371,2020-08-05 05:39:30+00:00,22.995433,114.299766,0,,3.4,202008 822649e2d142a486,2020-08-05 05:44:30+00:00,22.995433,114.299766,0,,3.4,202008 11115304371,2020-08-05 05:44:53+00:00,22.995433,114.299766,0,,3.4,202008 11115304371,2020-08-05 05:45:40+00:00,22.995433,114.299766,0,,5.8,202008 And the info.csv data likes this: car_id,device_id,car_type,car_num,marketer_name 1,11110110037,1,AAA,T1 2,11115304371,1,BBB,T2 3,11111100345,1,CCC,T3 4,11111100242,1,DDD,T4 5,12221100034,1,EEE,T5 6,12221100230,1,FFF,T6 7,14465301234,1,GGG,T7 When I use this code to merge 2 dataframe. import pandas as pd […]

Categories
Linux Mastering Development

How can I extract two data fields (1 scalar & 1 array) per node out of a very large (> 100,000 lines) JSON file?

I’ve got a 139,000 line JSON file, whose structure basically looks like this (it’s an extract from OpenStreetMap): { "type": "FeatureCollection", "generator": "overpass-ide", "features": [ { "type": "Feature", "properties": { "@id": "relation/7859", "TMC:cid_58:tabcd_1:Class": "Area", "TMC:cid_58:tabcd_1:LCLversion": "9.00", "TMC:cid_58:tabcd_1:LocationCode": "4934", "leisure": "park", "name": "Platnersberg", "type": "multipolygon", "@geometry": "center" }, "geometry": { "type": "Point", "coordinates": [ 11.128184, 49.4706035 […]

Categories
Mastering Development

MySQL: index for query with range and order by

I’m trying to get a query containing a range and ORDER BY … DESC to use an index. The index is used if I remove the ORDER BY and just use the range. The index is also used if I remove the range and just use the ORDER BY. Table: CREATE TABLE `messages` ( `id` […]

Categories
Mastering Development

Dataframe to JSON with NaN values

I want to convert the following data frame to JSON using groupby. ID,Location,Country,Latitude,Longitude,tide 8,WISMAR 2,DEU,53.8,11.4,69.0 8,WISMAR 2,DEU,53.8,11.4,68.0 10,SAN FRANCISCO,USA,37.8,-122.4,NaN Something like this: [ { "Country": "DEU", "ID": 8, "Latitude": 53.8, "Location": "WISMAR 2", "Longitude": 11.4, "Tide-Data": [69, 68] }, { "Country": "USA", "ID": 10, "Latitude": 37.8, "Location": "SAN FRANCISCO", "Longitude": -122.4, "Tide-Data": [] } The […]

Categories
Mastering Development

IF/ELSE conditions on Spark / JAVA

I’m trying to run the following code : for (java.util.Iterator<Row> iter = dataframe1.toLocalIterator(); iter.hasNext();){ Row it = (iter.next()); String item = it.get(2).toString(); String rayon = it.get(6).toString(); Double d=Double.parseDouble(rayon)/100000; String geomType = it.get(14).toString(); Dataset<Row> res_f = null; if(geomType.equalsIgnoreCase("Polygon")){ res_f= dataframe2.withColumn("ST_WITHIN",expr("ST_WITHIN(ST_GeomFromText(CONCAT(‘POINT(‘,longitude,’ ‘,latitude,’)’,4326)),ST_GeomFromWKT(‘"+item+"’))")); } else { res_f = dataframe2.withColumn("ST_BUFFERR",expr("ST_Buffer(ST_GeomFromWKT(‘"+item+"’),"+d+")")).withColumn("ST_WITHIN",expr("ST_WITHIN(ST_GeomFromText(CONCAT(‘POINT(‘,longitude,’ ‘,latitude,’)’,4326)),ST_BUFFERR)")); } res_f.show(); } But res_f returns nothing and […]

Categories
User Help

Dell Optiplex 7010 CPU transplant [closed]

The goal is to upgrade an Optiplex Core-i53570 with a Core i7-3770 via transplant. If this is possible, what are the obstacles one must overcome? BIOS? Other hardware? This is not a shopping request: I do not seek nor request hardware recommendations. The hardware listed below is in possession. I seek to understand what is […]

Categories
CMS Magento Mastering Development

Magento 2: How to work with larger collection of data that the site shouldn’t load too much?

I’m using this store locator extension in my 2.3.1 local. I have 1000 stores in the backend imported by using CSV. On the frontend, while I search that store address it should find the stores from that 1000 store which is working fine but it’s taking a very long time to show on google map. […]

Categories
Mastering Development Software

MVC Patterns: Is There a Design Pattern For Returning Lists (Arrays) of Complex Entity Objects?

I’m struggling with how I should build out my entities, services, and DAOs in an MVC web application when it comes to returning a list (array) of data. On one hand, returning an array of entity objects keeps the code lean and easy to maintain, however, there’s a performance hit for building and returning complex […]

Categories
Mastering Development

How to plot West & East State magnitudes using Bubble Map

I don’t have any plots for ‘mag’ showing on the Bubble Map. I’ve looked at Plotly’s examples, but not much explanation is given on how to reproduce same plot (https://plotly.com/python/bubble-maps/#united-states-bubble-map) Also how can I show only some states ie: West and East coast versus all state magnitudes appearing on plot? Sample data: state latitude longitude […]