Difference between revisions of "Guides:MAME - Troubleshooting"
Line 170: | Line 170: | ||
=== Check your buildenv versiom(s) === | === Check your buildenv versiom(s) === | ||
− | I currently use gcc, other tools are available (see MAME Documentation [https://docs.mamedev.org/initialsetup/compilingmame.html Compiling ]) | + | I currently use gcc, other tools are available (see MAME Documentation [https://docs.mamedev.org/initialsetup/compilingmame.html Compiling ] and [https://github.com/mamedev/buildtools Github]) |
<nowiki> | <nowiki> | ||
Line 280: | Line 280: | ||
(gdb) | (gdb) | ||
</nowiki> | </nowiki> | ||
− | |||
View a backtrace of the failure with the bt command | View a backtrace of the failure with the bt command | ||
Line 322: | Line 321: | ||
$ valgrind --track-origins=yes --tool=memcheck --leak-check=yes --show-reachable=yes --track-fds=yes --log-file=$HOME/logs/valgrind-$(date +'%s').log ./mamed -numscreens 2 -video opengl -window | $ valgrind --track-origins=yes --tool=memcheck --leak-check=yes --show-reachable=yes --track-fds=yes --log-file=$HOME/logs/valgrind-$(date +'%s').log ./mamed -numscreens 2 -video opengl -window | ||
</nowiki> | </nowiki> | ||
− | |||
− | |||
=== mtrace === | === mtrace === |
Revision as of 22:31, 30 October 2021
Troubleshooting MAME
MAME is a beast these days and when something doesn't work it can be difficult to know where to start, I am going to document some methods here, trying to help out on github issues. This is a teach a man to fish scenario.
Searching for Information
This will cover the simplest forms of finding information
Audit your roms
If you are receiving messages at launch about missing roms, this is the time to understand that roms evolve with MAME, i.e. dumps may be renamed or redumped over time. If you have downloaded a rom you found linked off a site it may be for an older version of MAME than what you have downloaded. Unless the site lists the compatible MAME version there is no way to tell _which_ version of MAME the set may be compatible with.
MAME however includes are database of compatible titles with each version either in the form of the main listxml output or a software list xml located in the hash directory. These databases contain the correct naming, set collections and file hashes to allow us and MAME to audit the rom you are trying to use.
These databases can be combined with tools like CLRMAME Pro (recommended) or | Romcenter to audit your romset and confirm its compatibility against each version.
Check your installation
Is your installation complete and up to date? Missing files or folders may be the cause of random issues
e.g. this cryptic message was the result of missing files from the bgfx folder which were discovered to during a git bisect
by the reporter
MESA-INTEL: warning: Performance support disabled, consider sysctl dev.i915.perf_stream_paranoid=0 X Error of failed request: BadDrawable (invalid Pixmap or Window parameter) Major opcode of failed request: 149 () Minor opcode of failed request: 4 Resource id in failed request: 0x700000e Serial number of failed request: 285 Current serial number in output stream: 290
Check the emulation status
MAME tells you the emulation status of each title in the UI (at the bottom of the screen and info tab) and at launch when using a frontend, if you have turned on skip_gameinfo in the ini file turn that off while troubleshooting
Confirm the emulation status if you are troubleshooting a machine or title to ensure it is not still considered work in progress
Read the documentation
MAME provides documentation in the forms listed below (nb: see the Documentation menu on mamedev.org)
Check social media
Check your general social media platforms for solutions to issues, you may find both community knowledge and devs floating around these platforms (nb: see the Community menu on mamedev.org)
Official
Unofficial
MAMETesters
MAMETesters is used as a community bug reporting tool more-so aligned to testing and reporting on specific title performance. You may search freely but will need an account to submit
github issues
Start your search with the github issues list, it may have been solved in a later code change or simply be documented there
Basic Troubleshooting
Simple troubleshooting with any type of build, some of these may be obvious but we will document them anyway
One step at a time
Make small change(s) to help identify the change that influenced the fix, you can report on this and help MAME progress. If you jump forward and change a lot document what was changed then backtrack until you break it again to try and isolate the problem.
Run in verbose mode
Run mame in -verbose mode and review the output
-verbose
Test an older version
Download (or build) and older version of MAME and test your configuration, find a build that works and report that to the MAME team, this can help identify a subset of changes that may be to blame.
Clean installation
Run with a clean installation, move your $HOME/.mame folder and run mame without the influences of any pre-existing configuration to see if the problem persists
BGFX
If troubleshooting BGFX issues, try a different chain and backend, up to version 0.236 the vulkan backend has been particularly troublesome whereas the more mature opengl backend is OK
Project
BGFX is a standalone project which is used by MAME to do the fancy display simulation but you already knew this.
- Take a look for known project issues
- Read the official documentation
Available Chains
You can see the available chains under ./bgfx/chains/ they are json files and will be called on the cli without the .json extension
UI Controls
See the sliders menu on the ingame menu (default key: tab) to change between chains
Consider your graphics stack
Look at your graphics stack,
- what does it comprise?
- Take into account any known issues with it
Include information in your graphics stack as part of you ticket.
Command Line Options
The cli options you will want are
-video bgfx -bgfx_backend (vulkan|opengl) -bgfx_chain <chain name>
Identify the running driver
Understand the impacted driver if you are looking at sf2ce that runs on the cps1 driver, you can identify this in your report
mame -listxml | awk '/name="sf2ce"/{print $3}' sourcefile="cps1.cpp">
If you can launch the game refer to the Driver: entry on the game info screen
Further Troubleshooting
Compiling
Compile your own build, this will give you the control you need to start narrowing down when the issue first presented.
This is a large topic please see the official documentation
I currently compile with
-j12 STRIP_SYMBOLS=1 NOWERROR=1 CFLAGS='-U_FORTIFY_SOURCE' TOOLS=1 REGENIE=1
Option | Usage | Reference |
---|---|---|
j12 | Threaded compile, can use up to (n)+1 cores | - |
STRIP_SYMBOLS=1 | Do not include debugging symbols | - |
NOWERROR=1 | Disable Warnings as Errors | bannister.org |
CFLAGS='-U_FORTIFY_SOURCE' | This is a compiler flag that is required to work around particulars with gcc | bannister.org Jira |
TOOLS=1 | Build external tools (i.e. chdmam, floptool) | - |
REGENIE=1 | Regenerate settings for build | - |
Compile a single driver
I can be quicker while troubleshooting to compile a single driver instead of everything
Shaders
Compile the shaders, not all shaders are compiled at any time so if you are testing CRT simulation issues compile the latest shaders
make shaders
Debug Symbols
Symbols help to give legible/useful output when debugging mame with an external debugger, retaining symbols will result in a larger executable, to retain symbols set STRIP_SYMBOLS to 0 when compiling (the default)
STRIP_SYMBOLS=0
Debug Builds
Once you have learnt to compile your own builds, you can now have the opportunity of building a debug version, to build a debug version pass DEBUG=1 to your build flags
DEBUG=1
This will build mamed (note the d)
When run you can enabled the debugger with -debug on the command line
Check your buildenv versiom(s)
I currently use gcc, other tools are available (see MAME Documentation Compiling and Github)
$ gcc --version gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0 Copyright (C) 2019 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Timelines
Look at the release date of the version you have identified as problematic and/or the settings which cause the issue, take that release date and have a look at the changes to the source code around at this time, was there updates to the feature you are having issues with?
Review the source code
Pull down a copy of the source code and start to take a look over it, perhaps by now you have identified a particular file that was changed for a feature where you have identified a crash, perhaps you just have an error message.
grep the source code for the error, what does the code seem to do? Trace this back through files until you are either completely lost or find what you are looking for.
Use the cppreference to try and understand what is going.
External Tools
git
git is the revision control tool used by the project
bisect
TBA
gdb
Debugging with gdb can be useful to get information (at a minimum a backtrace) to include in you issue report.
Missing Symbols
If you have not compiled mame with symbols do so now, you will be able to tell if you do not have symbols enabled in your build if you get a message when started
gdb ./mame --- snip --- Reading symbols from ./mame... (No debugging symbols found in ./mame)
or your backtrace looks like this note the "in ??"
Thread 1 "mame" received signal SIGSEGV, Segmentation fault. 0x00007ffff2444cb0 in ?? () from /lib/x86_64-linux-gnu/libGLX_mesa.so.0 (gdb) bt #0 0x00007ffff2444cb0 in ?? () from /lib/x86_64-linux-gnu/libGLX_mesa.so.0 #1 0x00007ffff24331d9 in ?? () from /lib/x86_64-linux-gnu/libGLX_mesa.so.0 #2 0x00007ffff7f25331 in ?? () from /lib/x86_64-linux-gnu/libSDL2-2.0.so.0 #3 0x00007ffff7e9d9cf in ?? () from /lib/x86_64-linux-gnu/libSDL2-2.0.so.0 #4 0x000055555ecaf0bf in ?? () #5 0x000055555ecaf1ad in ?? () #6 0x000055555ec79ad7 in ?? () #7 0x000055555ec82be1 in ?? () #8 0x000055555ec82eee in ?? () #9 0x000055555ec825f8 in ?? () #10 0x000055555ec78c99 in ?? () #11 0x0000555562d1a9be in ?? () #12 0x000055555ed6d573 in ?? () #13 0x0000555562c57ead in ?? () #14 0x0000555562c5944e in ?? () #15 0x000055555ed70729 in ?? () #16 0x000055555ee1c09e in ?? () #17 0x000055555ee1c332 in ?? () #18 0x000055555ed6e16a in ?? () #19 0x000055555a08212f in main () (gdb)
Start Using gdb
Start gdb with mame, loading mamed directly will auto load the symbols
gdb ./mamed --- snip --- Reading symbols from ./mame...
Confirm mame is loaded with list
(gdb) list 1 // license:BSD-3-Clause 2 // copyright-holders:Aaron Giles 3 /*************************************************************************** 4 5 mame.cpp 6 7 Specific (per target) constants 8 9 ****************************************************************************/ 10 (gdb)
Run mame in windowed mode from within gdb (any mame params will work here so use as you like)
run -window
Perform the actions you need to make MAME fail for debugging and the gdb will halt execution
--Type <RET> for more, q to quit, c to continue without paging--
Press the Return to drop to a prompt
Thread 1 "mamed" received signal SIGBUS, Bus error. 0x00007ffff242f776 in ?? () from /lib/x86_64-linux-gnu/libGLX_mesa.so.0 (gdb)
View a backtrace of the failure with the bt command
Thread 1 "mamed" received signal SIGBUS, Bus error. 0x00007ffff242f776 in ?? () from /lib/x86_64-linux-gnu/libGLX_mesa.so.0 (gdb) bt #0 0x00007ffff242f776 in ?? () from /lib/x86_64-linux-gnu/libGLX_mesa.so.0 #1 0x00007ffff2444c96 in ?? () from /lib/x86_64-linux-gnu/libGLX_mesa.so.0 #2 0x00007ffff24331d9 in ?? () from /lib/x86_64-linux-gnu/libGLX_mesa.so.0 #3 0x00007ffff7f25331 in ?? () from /lib/x86_64-linux-gnu/libSDL2-2.0.so.0 #4 0x00007ffff7e9d9cf in ?? () from /lib/x86_64-linux-gnu/libSDL2-2.0.so.0 #5 0x000055555f6a732f in renderer_sdl1::~renderer_sdl1() () #6 0x000055555f6a741d in renderer_sdl1::~renderer_sdl1() () #7 0x000055555f670ea7 in sdl_window_info::~sdl_window_info() () #8 0x000055555f67a4a9 in sdl_event_manager::process_window_event(running_machine&, SDL_Event&) () #9 0x000055555f67a7ee in sdl_event_manager::process_events(running_machine&) () #10 0x000055555f679e88 in sdl_osd_interface::poll_inputs(running_machine&) () #11 0x000055555f66f91e in sdl_osd_interface::input_update() () #12 0x0000555563a0f507 in video_manager::frame_update(bool) () #13 0x000055555f769253 in mame_machine_manager::create_ui(running_machine&) () #14 0x000055556394374d in running_machine::start() () #15 0x0000555563944d00 in running_machine::run(bool) () #16 0x000055555f76c549 in mame_machine_manager::execute() () #17 0x000055555f81c2ff in cli_frontend::start_execution(mame_machine_manager*, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) () #18 0x000055555f81c592 in cli_frontend::execute(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&) () #19 0x000055555f769f2a in emulator_info::start_frontend(emu_options&, osd_interface&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&) () #20 0x000055555a08697f in main () at ../../../../../src/mame/mame.cpp:23 (gdb)
You probably noted in the above output we are missing symbols for some systems libraries (sdl and mesa) to debug further we will need to enable the appropriate symbols (see distro doco) for these libraries.
Include the backtrace output in your report to assist with further troubleshooting
valgrind
WIP:: Valgrind "is an instrumentation framework for building dynamic analysis tools." ... yep sure thing, if you are asked to run a Memory Analyser by one of the devs to help out this may be your tool. I am currently testing this after being asked the same | here
Currently testing with
$ valgrind --track-origins=yes --tool=memcheck --leak-check=yes --show-reachable=yes --track-fds=yes --log-file=$HOME/logs/valgrind-$(date +'%s').log ./mamed -numscreens 2 -video opengl -window
mtrace
WIP:: requires modifying MAME to include mtrace support(?)
Final thoughts
Don't stress
The MAMEdev team know this source, you may not and it is a huge project to jump into. Raise an issue with your findings, a dev may make suggestions on where to look next or you may have managed to do 70% of the investigation for them, they do appreciate it.
Own your ignorance and ask for help, it is a learning opportunity that may have long term impact.