Categories
Mastering Development

How to concatenate a pandas multi index column dataframe with a single index dataframe

I have the following piece of code where I’m trying to perform a group by aggregation on a pivot table and concatenate the resulting aggregation back into the pivot table dataframe. But I’m having issues joining tables of different levels. import pandas as pd data = [ ["alice", "school 1", "math", 95], ["alice", "school 1", […]

Categories
Mastering Development

R: y-axis disappears when using coord_trans

Data: structure(list(Doy = c(1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L, 9L, 10L, 1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L, 9L, 10L, 1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L, 9L, 10L, 1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L, 9L, 10L), Greenup = c(108, NA, 107, NA, 108, 108, NA, 108, NA, 108, […]

Categories
Linux Mastering Development

Output from Bonnie++ doesn’t match up

I just ran bonnie++ ten times to test the performance of a file server. After removing the verbose output this is what remains: format_version,bonnie_version,name,concurrency,seed,file_size,io_chunk_size,putc,putc_cpu,put_block,put_block_cpu,rewrite,rewrite_cpu,getc,getc_cpu,get_block,get_block_cpu,seeks,seeks_cpu,num_files,max_size,min_size,num_dirs,file_chunk_size,seq_create,seq_create_cpu,seq_stat,seq_stat_cpu,seq_del,seq_del_cpu,ran_create,ran_create_cpu,ran_stat,ran_stat_cpu,ran_del,ran_del_cpu,putc_latency,put_block_latency,rewrite_latency,getc_latency,get_block_latency,seeks_latency,seq_create_latency,seq_stat_latency,seq_del_latency,ran_create_latency,ran_stat_latency,ran_del_latency 1.98,1.98,15.local,1,1602631781,8G,,8192,5,901,93,45205,13,23092,10,1162,90,66262,8,369.7,16,1,,,,100,18,1,360,4,21,0,18,1,542,4,20,0,12295us,665ms,1903ms,48384us,1944ms,5397ms,1652ms,13917us,2866ms,507ms,4953us,803ms 1.98,1.98,15.local,1,1602631781,8G,,8192,5,891,93,46620,14,23701,10,1177,90,66461,8,674.0,15,1,,,,100,18,1,406,4,22,0,18,1,559,4,23,0,12010us,1415ms,1264ms,52618us,496ms,69643us,1267ms,10064us,515ms,958ms,4650us,829ms 1.98,1.98,15.local,1,1602631781,8G,,8192,5,909,93,47824,14,21431,9,1147,88,67050,8,754.3,17,1,,,,100,20,1,400,4,19,0,14,1,530,3,20,0,11888us,1177ms,2710ms,45197us,1021ms,127ms,398ms,4562us,2874ms,1317ms,4810us,1006ms 1.98,1.98,15.local,1,1602631781,8G,,8192,5,890,91,37540,12,21765,9,1183,90,66822,8,1419,22,1,,,,100,19,1,461,3,23,0,19,1,547,3,19,0,11992us,548ms,864ms,46664us,480ms,25566us,760ms,44844us,1036ms,704ms,4778us,6449ms 1.98,1.98,15.local,1,1602631781,8G,,8192,5,795,83,43425,13,23066,10,1218,93,938042,27,1282,17,1,,,,100,19,1,406,4,21,0,18,1,560,4,23,0,53141us,14308ms,2786ms,39855us,71572us,6216ms,1761ms,6592us,1224ms,511ms,4654us,504ms 1.98,1.98,15.local,1,1602631781,8G,,8192,5,870,91,37339,11,21559,9,1187,91,924835,26,1499,19,1,,,,100,20,1,401,5,25,1,19,1,545,4,20,0,11615us,907ms,1870ms,44530us,352ms,1694us,626ms,4660us,359ms,659ms,5166us,972ms 1.98,1.98,15.local,1,1602631781,8G,,8192,5,881,93,37399,11,22194,9,1199,92,66756,8,1341,21,1,,,,100,17,1,409,4,20,0,17,1,557,4,23,0,11831us,1334ms,1113ms,45929us,579ms,29992us,4185ms,4348us,985ms,521ms,4889us,442ms 1.98,1.98,15.local,1,1602631781,8G,,8192,5,879,92,37592,12,21603,9,1208,93,4320730,99,1485,18,1,,,,100,17,1,403,4,22,0,16,1,570,3,19,0,11683us,665ms,1118ms,23421us,158us,5495ms,1623ms,4401us,592ms,863ms,10107us,1072ms 1.98,1.98,15.local,1,1602631781,8G,,8192,5,879,92,37386,11,21728,9,1168,90,554500,18,1516,19,1,,,,100,19,1,410,4,25,1,18,1,553,3,27,1,11325us,1617ms,1546ms,42335us,476ms,1812us,681ms,4294us,357ms,870ms,4776us,261ms 1.98,1.98,15.local,1,1602631781,8G,,8192,5,850,92,37915,11,21600,9,1159,88,141678,10,1513,20,1,,,,100,20,1,403,5,22,0,16,1,604,3,22,0,12674us,1461ms,1496ms,62742us,382ms,123ms,786ms,5874us,690ms,956ms,5011us,795ms but pasting this into a spread sheet results in missing columns (there are two header columns fewer than there are data […]

Categories
Mastering Development

Fast way to remove array of specific row values from 2D numpy array

I have a 2D array like this: a = np.array([[25, 83, 18, 71], [75, 7, 0, 85], [25, 83, 18, 71], [25, 83, 18, 71], [75, 48, 8, 43], [ 7, 47, 96, 94], [ 7, 47, 96, 94], [56, 75, 50, 0], [19, 49, 92, 57], [52, 93, 58, 9]]) and I want to […]

Categories
Mastering Development

Object.keys(myObject) returns an arbitrary list of numbers, not expected value

This is a bit of a head-scratcher – and has been a source of pain for the past 3 hours. Here’s my code. const payloadBody = JSON.parse(gatewayEvent.body); console.log(payloadBody); console.log(gatewayEvent.body); console.log(payloadBody.paymentMethodId); console.log(payloadBody[‘paymentMethodId’]); console.log(Object.keys(payloadBody)); If I look at the CloudWatch logs for this (I am executing in Lambda), the line console.log(payloadBody) logs: { "customerId": "cus_REDACTED", "paymentMethodId": "pm_REDACTED", […]

Categories
User Help

How to use FILTER function in excel to to exclude certain values?

First, this is becoming very difficult for me to explain, so please bear with me. I want to make it easier to calculate for a puzzle (killer sudokus basically). So I made an excel table for all possible permutations possible with 1 to 9, their sum, and their number of digits. Now I am trying […]

Categories
Artificial Intelligence (AI) Mastering Development

How should data be pre-processed in walk forward validation?

I have came across with different examples referring to the use of walk forward validation on regression problems. I wonder how the data should be prepared. For example: First example: Including the validation data in each window Training Validation Inputs Outputs Inputs Outputs 1,2,3 4,5,6 4,5,6 7,8,9 2,3,4 5,6,7 5,6,7 8,9,10 3,4,5 6,7,8 6,7,8 9,10,11 […]

Categories
Mastering Development

Efficient neighbourhood search in numpy ndarray instead of nested conditional for loops

Although there are many instances of the question: "What is the numpy alternative to nested for loops", I was unable to fins a suitable answer for my case. Here it goes: I have a 3D numpy array with "0" background and other integers as foreground. I would like to find and store the foreground voxels […]

Categories
Mastering Development

How to properly plot a bar chart in a Pandas dataframe? The x-values keep showing up as the index

I have a small dataframe with student_id, exam_1, exam_2, exam_3, exam_4, and exam_5 as columns. There are 5 students as well for the rows. What I’d like to do is plot a bar graph showing the exam grades of one student aka one specific row, and ultimately doing it for each or a specific student […]

Categories
Mastering Development

MySQL query for multi-column distinct plus an ancillary column condition

Imagine a flat table that tracks game matches in which each game has three participants: an attacker, a defender and a bettor who is wagering on the outcome of the battle between players 1 and 2. The table includes the names of the players and the bettor of each game, as well as the date […]