Google

Each sub-matrix $$\mathbf{W}_h^k$$ within a larger weight matrix $$\mathbf{W}_h$$—which is constructed by concatenating $$M$$ sub-matrices—has dimensions of $$d \times \frac{d_h}{M}$$. In this specification, $$d$$ denotes the number of rows, $$d_h$$ is the total number of columns for the complete matrix $$\mathbf{W}_h$$, and $$M$$ is the total count of sub-matrices. This guarantees that joining the $$M$$ sub-matrices horizontally results in the combined matrix having exactly $$d_h$$ columns.

Shape of a Concatenated Weight Sub-Matrix ($$\mathbf{W}_h^k$$)

A large matrix with 512 rows and 2048 columns is constructed by horizontally joining 8 smaller matrices of identical size. What are the dimensions (rows × columns) of each of the 8 smaller matrices?

A large weight matrix with dimensions 768 rows by 3072 columns is constructed by horizontally joining a number of smaller, identical sub-matrices. If each of these smaller sub-matrices has 256 columns, how many sub-matrices were joined together to form the large matrix?

Calculating the Number of Concatenated Sub-Matrices

Based on the rules of matrix concatenation, evaluate the engineer's plan. Is this plan feasible? Provide a clear justification for your conclusion.

Learn Before

Related