Position of layer normalization in transformer model

In Attention Is All You Need paper: That is, the output of each sub-layer is $LayerNorm(x+Sublayer(x))$, where $Sublayer(x)$ is the function implemented by the sub-layer itself. We apply dropout to the output of each sub-layer, before it is added to the sub-layer input and normalized. which makes the final formula $LayerNorm(x+Dropout(Sublayer(x)))$. However, in https://github.com/tensorflow/models/blob/0effd158ae1e6403c6048410f79b779bdf344d7d/official/transformer/model/transformer.py#L278-L288, I…

Details

reading in a file from ubuntu on local machine?

I have a python script which I’m running on AWS (EC2 instance with Ubuntu). This python script outputs a JSON file daily, to a directory in /home/ubuntu: with open(“/home/ubuntu/bandsintown/sf_events.json”, “w”) as writeJSON: file_str = json.dumps(allEvents, sort_keys=True) file_str = “var sf_events = ” + file_str All works as expected here. My issue is that I’m unsure…

Details