Deep Learning Tools by hugofloresgarcia · Pull Request #1384 · audacity/audacity

hugofloresgarcia · 2021-07-29T17:53:45Z

Adds source separation and deep learning tools to audacity.

I signed CLA
I made sure the code compiles on my machine
I made sure there are no unnecessary changes in the code
I made sure the title of the PR reflects the core meaning of the issue you are solving
I made sure the commit message(s) contain a description and answer the question "Why do those changes fix that particular issue?" or "Why are those changes really necessary as improvements?"

crsib

I've just quickly run through :)

As an addition to other comments - there are some files without an empty line at the end of the file.

.gitignore

CMakeLists.txt

src/CMakeLists.txt

src/effects/deeplearning/DeepModel.h

src/effects/deeplearning/DeepModelManager.h

src/effects/deeplearning/ModelCard.cpp

src/effects/deeplearning/ModelCard.h

crsib

Also, quite a few files still miss new line in the end of file

CMakeLists.txt

src/effects/deeplearning/EffectDeepLearning.cpp

src/effects/deeplearning/ModelCard.cpp

src/effects/deeplearning/ModelCard.h

crsib · 2021-08-11T10:43:00Z

Also, it looks like we should only allow Source Separation in Win64 builds

crsib

What should be improved:

Consistent use of const and final
I would preffer having *Ptr instead of *Holder
Translatable strings should have hints for translators
Error handling should be more consistent. Important context information should not be lost
!= instead of !(=) whenver possible
In most cases - it is advised to have separate h/cpp for a class. Exceptions are ok to be groupped whith the class that throws it

src/effects/deeplearning/DeepModel.cpp

src/effects/deeplearning/ModelManagerPanel.cpp

src/effects/deeplearning/ModelManagerPanel.h

Paul-Licameli · 2022-01-13T00:32:43Z

@hugofloresgarcia I have not yet caught up in a second round of review of your anwsers to my suggestions -- let me know when the branch is stable and I should do it.

Meanwhile I have pushed two more suggested commits.

The first is a cherry-pick from the master, which will disappear from this branch if you rebase it onto latest master.

The second makes use of the new utility defined in the first, to simplify the sending of messages from ActiveModel, which is the class I introduced to break dependency cycles.

This is the well known "Observer Pattern" in action -- it is a great friend for the breaking of cycles, and I use it so often I wrote a new class template to help generate instances of it.

It often happens that something UI related, like the dialog here, needs to be at the higher level of the dependency graph, and so it is the "subscriber" to messages that are "published" by the lower level object (here ActiveModel) which ought to implement the core logic of the program and not have unnecessary dependencies on the UI code.

In well structured code, the flow of control often goes in the opposite direction to the static build-time dependency arrows, and indirect function calls through pointers make this possible.

By explicitly specifying the StartupWMClass, the elementaryOS Dock is now able to associate Audacity's window with the flatpak-launcher icon so that it does not spawn a second icon for Audacity.

- Adds libtorch (a deep learning library) via conan. - Adds instructions for building Audacity with Deep Learning features. - Adds deeplearning-models, a module with built-in deep learning models. - Adds mod-deep-learning, which contains source code files for Audacity's Deep Learning module. - Adds a DeepModel class, a container for using libtorch neural network models to process audio waveforms. - Adds a ModelCard class, a DTO for parsing deep model metadata stored in JSON format, with appropriate serialization/deserialization utilities

a singleton object for downloading and managing deep learning models, hosted in HuggingFace and installed locally.

to store the user's choice of model, and send notifications of its changes

an abstract class for building UI panels out of ModelCards, and two derived classes for displaying ModelCards two different ways (compact and detailed).

Deep Learning Effect and Deep Learning Analyzer. Deep Learning Effect performs waveform to waveform processing, and is useful for audio-in-audio-out tasks (such as source separation, voice conversion, style transfer, amplifier emulation, etc.), while Deep Learning Analyzer performs waveform to labels processing, and is useful for annotation tasks (such as sound event detection, musical instrument recognition, automatic speech recognition, etc.).

This simplifies the interaction between the ModelManager and UI objects, since they can just wait for ModelCards to finish fetching. Adds threading capabilities to the model manager via ThreadPool Adds a find function to ModelCardCollection Removes ModelSizeCallbacks from ModelManagerPanel, since all of this is Handled by the ModelManager now. Removes card fetch status bar

CopyLibs was invoked for modules only on macOS. However, if the module depends on different set of libraries - Audacity will fail to load it.

chrahunt · 2022-02-13T16:42:17Z

BUILDING.md


+### Building with Deep Learning 
+
+To enable deep learning features, pass `audacity_has_deep_learning=on` and `audacity_has_networking=yes` to CMake. 


Should this be audacity_has_deeplearning?

LWinterberg · 2024-10-15T07:53:40Z

Hello! We just switched master to 4.0.0 which makes this PR incompatible and/or obsolete as we're switching UI framework and reorganizing large parts of the codebase.

hugofloresgarcia force-pushed the deeplearning branch 3 times, most recently from 2717ab8 to eeefb21 Compare August 2, 2021 21:41

crsib reviewed Aug 4, 2021

View reviewed changes

hugofloresgarcia force-pushed the deeplearning branch 13 times, most recently from b5256dd to 617ff86 Compare August 10, 2021 18:29

crsib suggested changes Aug 11, 2021

View reviewed changes

hugofloresgarcia force-pushed the deeplearning branch 6 times, most recently from 5bdb211 to ff6016c Compare August 17, 2021 00:33

hugofloresgarcia marked this pull request as ready for review August 17, 2021 15:30

hugofloresgarcia force-pushed the deeplearning branch 3 times, most recently from db2d696 to f1371cc Compare August 17, 2021 21:25

crsib suggested changes Aug 18, 2021

View reviewed changes

hugofloresgarcia force-pushed the deeplearning branch 4 times, most recently from fa2f0af to 2285a28 Compare January 14, 2022 17:00

Add StartupWMClass to .desktop file

b06e9f6

By explicitly specifying the StartupWMClass, the elementaryOS Dock is now able to associate Audacity's window with the flatpak-launcher icon so that it does not spawn a second icon for Audacity.

hugofloresgarcia force-pushed the deeplearning branch 2 times, most recently from 4858f99 to ce3989d Compare January 18, 2022 01:56

hugofloresgarcia added 9 commits January 18, 2022 12:27

Adds DeepModelManager

f537698

a singleton object for downloading and managing deep learning models, hosted in HuggingFace and installed locally.

Adds Active Model,

49de4bf

to store the user's choice of model, and send notifications of its changes

Adds ModelCardPanel

461cc23

an abstract class for building UI panels out of ModelCards, and two derived classes for displaying ModelCards two different ways (compact and detailed).

Adds a UI panel for navigating multiple ModelCardPanels

1d4a328

Adds an abstract class for deep learning audio processors

314e758

fixes CurlResponse's httpCode when the result is CURLE_OK

62f8d67

allow requests in lib-network-manager to block the caller's thread

2fefe22

hugofloresgarcia force-pushed the deeplearning branch from ce3989d to 7edccc3 Compare January 18, 2022 18:27

hugofloresgarcia added 4 commits January 18, 2022 13:37

use struct to avoid dangling references in fwd pass

59e9bb3

add sorting to modelcardcollection

e5b9c29

use uniform initialization with exceptions

8fd007f

hugofloresgarcia force-pushed the deeplearning branch from 721268f to 8fd007f Compare January 18, 2022 19:37

crsib and others added 2 commits January 26, 2022 11:02

Perform the libraries related magic for modules on all platforms

9097141

CopyLibs was invoked for modules only on macOS. However, if the module depends on different set of libraries - Audacity will fail to load it.

[ModelManagerPanel] add wxTAB_TRAVERSAL flag on ManagerToolsPanel

5a6f239

chrahunt reviewed Feb 13, 2022

View reviewed changes

fix has_deeplearning flag on Building instructions

af6928e

LWinterberg removed a link to an issue Jan 24, 2023

AI Vocal Remover #718

Closed

LWinterberg closed this Oct 15, 2024


		### Building with Deep Learning

		To enable deep learning features, pass `audacity_has_deep_learning=on` and `audacity_has_networking=yes` to CMake.

Conversation

hugofloresgarcia commented Jul 29, 2021

Uh oh!

crsib left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

crsib left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

crsib commented Aug 11, 2021

Uh oh!

crsib left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Paul-Licameli commented Jan 13, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chrahunt Feb 13, 2022

Choose a reason for hiding this comment

Uh oh!

LWinterberg commented Oct 15, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Paul-Licameli commented Jan 13, 2022 •

edited

Loading