It kinda works with m2 mps device

[cameronbergh]

Heyo, so i ran this on my 2023 m2 macbook and got some results. it uses the gpu but doesnt quite get it right.

what i said into the microphone was
"hi hows it going"
"whats up"
"what it do"

anywhere here is my report:

(whisper_real_time) cameron@M2 whisper_real_time % pip freeze
certifi==2022.12.7
charset-normalizer==3.0.1
ffmpeg-python==0.2.0
filelock==3.9.0
future==0.18.3
huggingface-hub==0.12.1
idna==3.4
more-itertools==9.0.0
mpmath==1.2.1
networkx==3.0rc1
numpy==1.24.2
openai-whisper @ git+https://github.com/openai/whisper.git@51c785f7c91b8c032a1fa79c0e8f862dea81b860
packaging==23.0
PyAudio==0.2.13
PyYAML==6.0
regex==2022.10.31
requests==2.28.2
SpeechRecognition==3.9.0
sympy==1.11.1
tokenizers==0.13.2
torch==2.0.0.dev20230121
torchaudio==2.0.0.dev20230223
tqdm==4.64.1
transformers==4.26.1
typing_extensions==4.5.0
urllib3==1.26.14

(whisper_real_time) cameron@M2 whisper_real_time % python transcribe_demo.py --model large --non_english
Model loaded.

/Users/cameron/.local/share/virtualenvs/whisper_real_time-Iw30K9az/lib/python3.9/site-packages/whisper/decoding.py:633: UserWarning: The operator 'aten::repeat_interleave.self_int' is not currently supported on the MPS backend and will fall back to run on the CPU. This may have performance implications. (Triggered internally at /Users/runner/work/pytorch/pytorch/pytorch/aten/src/ATen/mps/MPSFallback.mm:11.)
audio_features = audio_features.repeat_interleave(self.n_group, dim=0)

Hi<|en|><|en|> Hi Hi Hi Hi Hi Hi Hi

Hi<|en|><|en|> Hi Hi Hi Hi Hi Hi Hi
What<|en|><|en|><|en|>

Hi<|en|><|en|> Hi Hi Hi Hi Hi Hi Hi
What<|en|><|en|><|en|>
What<|en|><|en|><|en|> What What
^C

Transcription:
Hi<|en|><|en|> Hi Hi Hi Hi Hi Hi Hi
What<|en|><|en|><|en|>
What<|en|><|en|><|en|> What What

[DenisVieriu97]

@cameronbergh these issues should be resolved in latest pytorch nightly - including repeat_interleave. Could you try a more recent version of torch, such as 2.0.0.dev20230224? You can get it using pip3 install --pre --force-reinstall torch --index-url https://download.pytorch.org/whl/nightly/cpu

[cameronbergh]

openai/whisper#382 (comment)