Categories
Development

Runtime error when reading data from a training dataset PyTorch

I have a sample of data in my training dataset which I am able to view if I print the data, but when accessing it to train the data, I keep getting RuntimeError: Expected object of scalar type Double but got scalar type Float for argument #2 ‘weight’ in call to _thnn_conv2d_forward. I am unable […]

Categories
Artificial Intelligence (AI) Development

Why my network overfits on the CIFAR-10 dataset?

I built the following network architecture: Net( (conv1): Sequential( (0): Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): ReLU() (2): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (3): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False) (4): Dropout(p=0, inplace=False)) (conv2): Sequential( (0): Conv2d(32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)) (1): ReLU() (2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True) (3): […]

Categories
Artificial Intelligence (AI) Development

Maximum and Minimum filters in a Convolution Layer

I’m trying to figure out how to write an optimal convolutional neural network with respect to maximizing and minimizing filters in a Convolution2D layer. This is my thinking and I’m not sure if its correct. If we have a dataset of 32×32 images, we could start with a Convolutional2D layer, filter of 3×3 and stride […]

Categories
Development

1 dimenstional convolution error in using tensorflow

I am studying 1d convolution using tensorflow. Code: import numpy as np import tensorflow as tf \#####raw data, input length is 24, and feature_len is 6 batch = np.ceil((np.random.rand(24, 6)*10))-5 \#####filter for convoltion, filter width is 3, filter input dim is 6, output dim is 18 eye_filter = tf.constant(np.eye(3*6).reshape(3,6,18).reshape(3,6,18)) \#####here error happened conv = tf.nn.conv1d(input=batch, […]

Categories
Development

How to implement LadderNet (2 U-Nets) in Keras? (With available PyTorch script as reference)

I am trying to implement the architecture of LadderNet (https://arxiv.org/abs/1810.07810) in Keras, with only the PyTorch version available as reference. The architecture in the paper is comprised of 2 U-Nets: The codes for the PyTorch implementation of LadderNet’s architecture (obtained from https://github.com/juntang-zhuang/LadderNet/blob/master/src/LadderNetv65.py) and Keras’ implementation of U-Net (obtained from https://github.com/zhixuhao/unet/blob/master/model.py) are respectively: drop = 0.25 […]

Categories
Development

RuntimeError: Expected 4-dimensional input for 4-dimensional weight [32, 4, 8, 8], but got 2-dimensional input of size [1, 4] instead

I am initializing a Convolutional DQN with the following code: class ConvDQN(nn.Module): def __init__(self, input_dim, output_dim): super(ConvDQN, self).__init__() self.input_dim = input_dim self.output_dim = output_dim self.conv = nn.Sequential( nn.Conv2d(self.input_dim, 32, kernel_size=8, stride=4), nn.ReLU(), nn.Conv2d(32, 64, kernel_size=4, stride=2), nn.ReLU(), nn.Conv2d(64, 64, kernel_size=3, stride=1), nn.ReLU() ) self.fc_input_dim = self.feature_size() self.fc = nn.Sequential( nn.Linear(self.fc_input_dim, 128), nn.ReLU(), nn.Linear(128, 256), nn.ReLU(), […]

Categories
Artificial Intelligence (AI) Development

What should the dimension of the mean and variance of the latent distribution in variational auto-encoders be?

I’m having a problem to understand the needed dimensions of an VAE, especially for mu, logvar and z layer. Let’s say I have an input of 512×512, 1 color channel (CT images), batch size 32. Then my encoder/decoder looks like the following: self.encoder = nn.Sequential( nn.Conv2d(1, 32, kernel_size=3, stride=1, padding=1), # 32x512x512 nn.ReLU(True), nn.Conv2d(32, 32, […]

Categories
Development

Getting errors with pytorch grouped convolutions

I am trying to use Pytorch grouped Conv2d operator on very large images (10k x 10k pixels). I am getting an RuntimeError: offset is too big error when trying to apply a grouped convolution in the network. Anyone knows how to circumvent this? Code for reproducibility: import torch import torch.nn as nn import pdb def […]

Categories
Development

Different output of two “same” padding CNN models in pytorch

When I realized PRnet(script1), The effect is very poor. So I compared the architecture of my model with that of a correct model(script3). I found the tensor xm(in script1.forward function) is the same as the xm(in script3.forward function). But the xn in script1 is different from script3. note: 1:The self.decoder in script1 is copied form […]

Categories
Development

Why does the pytorch model perform poorly after setting eval()?

I used pytorch to build a segmentation model that uses the BatchNormalization layer. I found that when I set model.eval() on the test, the test result will be 0. If I don’t set model.eval(), it will perform well. I tried to search for related questions, but I got the conclusion that model.eval() can fix the […]