Here, you can learn how to implement Bidirectional LSTM with Keras.

With a Bidirectional LSTM layer, you can see the improvement in accuracy with fewer epochs.

```
model = Sequential()
model.add(Embedding(len(char_indices)+1,
300,
batch_size=(seq_length-time_step)*batch_size,
weights=[embedding_matrix],
mask_zero=True,
trainable=False))
model.add(Bidirectional(LSTM(512,
dropout=0.15,
return_sequences=True),
merge_mode='concat'))
model.add(Bidirectional(LSTM(512,dropout=0.15),merge_mode='concat'))
model.add(Dense(len(char_indices)+1, activation='softmax'))
```

You should note that if the LSTM layer has `return_sequence=True`

, it cannot connect directly to the Dense layer.

The Dense layer can only accept two-dimensional sequences; when the LSTM layer does not have a `return_sequence`

, it outputs a two-dimensional sequences. However, if it has `return_sequence=True`

, it outputs a third-order sequences. As a result, an error occurs. For example,

`ValueError: Shapes (None, None) and (3500, None, 10500) are incompatible`

One way to solve this problem is to put another LSTM layer that does not have `return_sequence`

. This layer accepts the three-dimensional sequences from the previous LSTM layer and passes the two-dimensional sequences to the Dense layer.

In this way, the second LSTM layer seems to have little effect on the prediction results.