In new Rails 8 app, ActiveAdmin 4 (beta 15 or master), after raising an error such as NoMethodError in some AA resource class, triggers superclass mismatch for class DashboardController. The only mitigation is to restart the server.
The error needs to be raised at the "root" of the class/resource, such as:

ActiveAdmin.register Post do
  raise 'asd'
end

Here are the steps to repro:

rails new superclass_admin 
cd superclass_admin
bundle install
rails db:prepare
rails g scaffold post title:string
bundle add activeadmin --version "4.0.0.beta15"
rails g active_admin:install --skip-users
rails db:migrate
rails g active_admin:resource Post
rails s

After this, open app/admin/posts.rb and change permit_params :title to aaapermit_param :title.
You get undefined method 'aaapermit_params' for an instance of ActiveAdmin::ResourceDSL as expected.
Now change aaapermit_params back to permit_params and reload the page.

Expected behavior

The page should reload.

Actual behavior

superclass mismatch for class DashboardController

A quick patch that I discovered is to add this code to the gem:

# lib/active_admin/namespace.rb

    # TODO: replace `eval` with `Class.new`
    def register_resource_controller(config)
+     name_parts = config.controller_name.split('::')
+     const_name = name_parts.last
+     if Admin.const_defined?(const_name)
+       Admin.send(:remove_const, const_name)
+     end
      eval "class ::#{config.controller_name} < ActiveAdmin::ResourceController; end"
      config.controller.active_admin_config = config
    end