Yes absolutely, convolutions are the base for all the SOTA algorithms present right now. On the side note, ViT-L takes only 0.68K days to train which is a huge improvement from the RESNET152 with improved accuracies (except VTAB). Convolutions will stick around for a while for sure :)