Image for post
Image for post
Model Overview. The image is taken from the paper.

The Limitation with Transformers For Images


Image for post
Image for post
Underspecification in a simple epidemiological model. The image is taken from the paper.

Introduction & Overview


Image for post
Image for post
Overview of the proposed approach MAMA. MAMA constructs an open knowledge graph (KG) with a single forward pass of the pre-trained Language model (LM) (without fine-tuning) over the corpus. The image is taken from the paper.

Introduction & Overview

In this paper, the authors design an unsupervised approach called MAMA that successfully recovers the factual knowledge stored in Language Models to build Knowledge Graphs from scratch. MAMA constructs a KG with a single forward pass of a pre-trained LM (without fine-tuning) over a textual corpus. …


Image for post
Image for post
Approximation of the regular attention mechanism AV (before D⁻¹ -renormalization) via (random) feature maps. Dashed-blocks indicate the order of computation with corresponding time complexities attached. The image is taken from the paper.

Introduction & Overview


Image for post
Image for post
Comparison between attention and lambda layers. (Left) An example of 3 queries and their local contexts within a global context. (Middle) The attention operation associates each query with an attention distribution over its context. (Right) The lambda layer transforms each context into a linear function lambda that is applied to the corresponding query. The image is taken from the paper

Introduction & Overview

Lambda Layers Vs Attention Layers

About

Nakshatra Singh

A Machine Learning, Deep Learning, and Natural Language Processing enthusiast. I make research papers easy to read.😎❤️

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store