Difference between revisions of "Guides:MAME - Troubleshooting"

From Wah!ki
Jump to navigation Jump to search
Line 105: Line 105:
  
 
When run you can enabled the debugger with -debug on the command line and if using an external debugging tool like gdb you should have useful output in your backtrace
 
When run you can enabled the debugger with -debug on the command line and if using an external debugging tool like gdb you should have useful output in your backtrace
 
=== External Tools ====
 
* gdb
 
  
 
=== Timelines ===
 
=== Timelines ===
Line 116: Line 113:
  
 
grep the source code for the error, what does the code seem to do? Trace this back through files until you are either completely lost or find what you are looking for.
 
grep the source code for the error, what does the code seem to do? Trace this back through files until you are either completely lost or find what you are looking for.
 +
 +
 +
== External Tools ==
 +
=== gdb ===
 +
Debugging with gdb can be useful to get information (at a minimum a backtrace) to include in you issue report.
 +
 +
==== Missing Symbols ====
 +
If you have not compiled mame with symbols do so now, you will be able to tell if you do not have symbols enabled in your build if you get a message when started
 +
<nowiki>
 +
gdb ./mame
 +
--- snip ---
 +
Reading symbols from ./mame...
 +
(No debugging symbols found in ./mame)
 +
</nowiki>
 +
 +
or your backtrace looks like this note the "in ??"
 +
<nowiki>
 +
Thread 1 "mame" received signal SIGSEGV, Segmentation fault.
 +
0x00007ffff2444cb0 in ?? () from /lib/x86_64-linux-gnu/libGLX_mesa.so.0
 +
(gdb) bt
 +
#0  0x00007ffff2444cb0 in ?? () from /lib/x86_64-linux-gnu/libGLX_mesa.so.0
 +
#1  0x00007ffff24331d9 in ?? () from /lib/x86_64-linux-gnu/libGLX_mesa.so.0
 +
#2  0x00007ffff7f25331 in ?? () from /lib/x86_64-linux-gnu/libSDL2-2.0.so.0
 +
#3  0x00007ffff7e9d9cf in ?? () from /lib/x86_64-linux-gnu/libSDL2-2.0.so.0
 +
#4  0x000055555ecaf0bf in ?? ()
 +
#5  0x000055555ecaf1ad in ?? ()
 +
#6  0x000055555ec79ad7 in ?? ()
 +
#7  0x000055555ec82be1 in ?? ()
 +
#8  0x000055555ec82eee in ?? ()
 +
#9  0x000055555ec825f8 in ?? ()
 +
#10 0x000055555ec78c99 in ?? ()
 +
#11 0x0000555562d1a9be in ?? ()
 +
#12 0x000055555ed6d573 in ?? ()
 +
#13 0x0000555562c57ead in ?? ()
 +
#14 0x0000555562c5944e in ?? ()
 +
#15 0x000055555ed70729 in ?? ()
 +
#16 0x000055555ee1c09e in ?? ()
 +
#17 0x000055555ee1c332 in ?? ()
 +
#18 0x000055555ed6e16a in ?? ()
 +
#19 0x000055555a08212f in main ()
 +
(gdb)
 +
</nowiki>
 +
 +
==== Start Using gdb ====
 +
Start gdb with mame, loading mamed directly will auto load the symbols
 +
<nowiki>
 +
gdb ./mamed
 +
--- snip ---
 +
Reading symbols from ./mame...
 +
</nowiki>
 +
 +
Confirm mame is loaded with list
 +
<nowiki>
 +
(gdb) list
 +
1 // license:BSD-3-Clause
 +
2 // copyright-holders:Aaron Giles
 +
3 /***************************************************************************
 +
4
 +
5     mame.cpp
 +
6
 +
7     Specific (per target) constants
 +
8
 +
9 ****************************************************************************/
 +
10
 +
(gdb)
 +
</nowiki>
 +
 +
Run mame in windowed mode from within gdb (any mame params will work here so use as you like)
 +
<nowiki>
 +
run -window
 +
</nowiki>
 +
 +
Perform the actions you need to make MAME fail for debugging and the gdb will halt execution
 +
<nowiki>
 +
--Type <RET> for more, q to quit, c to continue without paging--
 +
</nowiki>
 +
 +
Press the Return to drop to a prompt
 +
<nowiki>
 +
Thread 1 "mamed" received signal SIGBUS, Bus error.
 +
0x00007ffff242f776 in ?? () from /lib/x86_64-linux-gnu/libGLX_mesa.so.0
 +
(gdb)
 +
</nowiki>
 +
 +
 +
View a backtrace of the failure with the bt command
 +
<nowiki>
 +
Thread 1 "mamed" received signal SIGBUS, Bus error.
 +
0x00007ffff242f776 in ?? () from /lib/x86_64-linux-gnu/libGLX_mesa.so.0
 +
(gdb) bt
 +
#0  0x00007ffff242f776 in ?? () from /lib/x86_64-linux-gnu/libGLX_mesa.so.0
 +
#1  0x00007ffff2444c96 in ?? () from /lib/x86_64-linux-gnu/libGLX_mesa.so.0
 +
#2  0x00007ffff24331d9 in ?? () from /lib/x86_64-linux-gnu/libGLX_mesa.so.0
 +
#3  0x00007ffff7f25331 in ?? () from /lib/x86_64-linux-gnu/libSDL2-2.0.so.0
 +
#4  0x00007ffff7e9d9cf in ?? () from /lib/x86_64-linux-gnu/libSDL2-2.0.so.0
 +
#5  0x000055555f6a732f in renderer_sdl1::~renderer_sdl1() ()
 +
#6  0x000055555f6a741d in renderer_sdl1::~renderer_sdl1() ()
 +
#7  0x000055555f670ea7 in sdl_window_info::~sdl_window_info() ()
 +
#8  0x000055555f67a4a9 in sdl_event_manager::process_window_event(running_machine&, SDL_Event&) ()
 +
#9  0x000055555f67a7ee in sdl_event_manager::process_events(running_machine&) ()
 +
#10 0x000055555f679e88 in sdl_osd_interface::poll_inputs(running_machine&) ()
 +
#11 0x000055555f66f91e in sdl_osd_interface::input_update() ()
 +
#12 0x0000555563a0f507 in video_manager::frame_update(bool) ()
 +
#13 0x000055555f769253 in mame_machine_manager::create_ui(running_machine&) ()
 +
#14 0x000055556394374d in running_machine::start() ()
 +
#15 0x0000555563944d00 in running_machine::run(bool) ()
 +
#16 0x000055555f76c549 in mame_machine_manager::execute() ()
 +
#17 0x000055555f81c2ff in cli_frontend::start_execution(mame_machine_manager*, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) ()
 +
#18 0x000055555f81c592 in cli_frontend::execute(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&) ()
 +
#19 0x000055555f769f2a in emulator_info::start_frontend(emu_options&, osd_interface&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&) ()
 +
#20 0x000055555a08697f in main () at ../../../../../src/mame/mame.cpp:23
 +
(gdb)
 +
</nowiki>
 +
 +
You probably noted in the above output we are missing symbols for some systems libraries (sdl and mesa) to debug further we will need to enabled the appropriate symbols for these libraries.
 +
 +
Include the backtrace output in your report to assist with further troubleshooting
  
 
== Final thoughts ==
 
== Final thoughts ==

Revision as of 20:57, 21 October 2021

Troubleshooting MAME

MAME is a beast these days and when something doesn't work it can be difficult to know where to start, I am going to document some methods here, trying to help out on github issues. This is a teach a man to fish scenario.

Searching for Infomation

This will cover the simplest forms of finding information

Read the documentation

MAME provides documentation in the forms listed below (nb: see the Documentation menu on mamedev.org)

Check social media

Check your general social media platforms for solutions to issues, you may find both community knowledge and devs floating around these platforms (nb: see the Community menu on mamedev.org)

Official

Unofficial

MAMETesters

MAMETesters is used as a community bug reporting tool more-so aligned to testing and reporting on specific title performance. You may search freely but will need an account to submit

github issues

Start your search with the github issues list, it may have been solved in a later code change or simply be documented there

Basic Troubleshooting

Simple troubleshooting with any type of build, some of these may be obvious but we will document them anyway

One step at a time

Make small change(s) to help identify the change that influenced the fix, you can report on this and help MAME progress. If you jump forward and change a lot document what was changed then backtrack until you break it again to try and isolate the problem.

Run in verbose mode

Run mame in -verbose mode and review the output

-verbose

Test an older version

Download (or build) and older version of MAME and test your configuration, find a build that works and report that to the MAME team, this can help identify a subset of changes that may be to blame.

Clean installation

Run with a clean installation, move your $HOME/.mame folder and run mame without the influences of any pre-existing configuration to see if the problem persists

BGFX

If troubleshooting BGFX issues, try a different chain and backend, up to version 0.236 the vulkan backend has been particularly troublesome whereas the more mature opengl backend is OK

You can see the available chains under ./bgfx/chains/ they are json files and will be called on the cli without the .json extension

The cli options you will want are

-video bgfx -bgfx_backend (vulkan|opengl) -bgfx_chain <chain name>

Alternatively see the sliders menu on the ingame menu (default: tab) to change between chains

Further Troubleshooting

Compiling

Compile your own build, this will give you the control you need to start narrowing down when the issue first presented.

This is a large topic please see the official documentation

I currently compile with

-j12 STRIP_SYMBOLS=1 NOWERROR=1 CFLAGS='-U_FORTIFY_SOURCE' TOOLS=1 REGENIE=1

Compile options
Option Usage Reference
j12 Threaded compile, can use up to (n)+1 cores -
STRIP_SYMBOLS=1 Do not include debugging symbols -
NOWERROR=1 Disable Warnings as Errors bannister.org
CFLAGS='-U_FORTIFY_SOURCE' This is a compiler flag that is required to work around particulars with gcc bannister.org Jira
TOOLS=1 Build external tools (i.e. chdmam, floptool) -
REGENIE=1 Regenerate settings for build -

Shaders

Compile the shaders, not all shaders are compiled at any time so if you are testing CRT simulation issues compile the latest shaders

make shaders

Debug Builds

Once you have learnt to compile your own builds, you can now have the opportunity of building a debug version, to build a debug version pass DEBUG=1 to your build flags

DEBUG=1 STRIP_SYMBOLS=0

This will build mamed (note the d) this is the executable you will run

When run you can enabled the debugger with -debug on the command line and if using an external debugging tool like gdb you should have useful output in your backtrace

Timelines

Look at the release date of the version you have identified as problematic and/or the settings which cause the issue, take that release date and have a look at the changes to the source code around at this time, was there updates to the feature you are having issues with?

Review the source code

Pull down a copy of the source code and start to take a look over it, perhaps by now you have identified a particular file that was changed for a feature where you have identified a crash, perhaps you just have an error message.

grep the source code for the error, what does the code seem to do? Trace this back through files until you are either completely lost or find what you are looking for.


External Tools

gdb

Debugging with gdb can be useful to get information (at a minimum a backtrace) to include in you issue report.

Missing Symbols

If you have not compiled mame with symbols do so now, you will be able to tell if you do not have symbols enabled in your build if you get a message when started

gdb ./mame
--- snip ---
Reading symbols from ./mame...
(No debugging symbols found in ./mame)

or your backtrace looks like this note the "in ??"

Thread 1 "mame" received signal SIGSEGV, Segmentation fault.
0x00007ffff2444cb0 in ?? () from /lib/x86_64-linux-gnu/libGLX_mesa.so.0
(gdb) bt
#0  0x00007ffff2444cb0 in ?? () from /lib/x86_64-linux-gnu/libGLX_mesa.so.0
#1  0x00007ffff24331d9 in ?? () from /lib/x86_64-linux-gnu/libGLX_mesa.so.0
#2  0x00007ffff7f25331 in ?? () from /lib/x86_64-linux-gnu/libSDL2-2.0.so.0
#3  0x00007ffff7e9d9cf in ?? () from /lib/x86_64-linux-gnu/libSDL2-2.0.so.0
#4  0x000055555ecaf0bf in ?? ()
#5  0x000055555ecaf1ad in ?? ()
#6  0x000055555ec79ad7 in ?? ()
#7  0x000055555ec82be1 in ?? ()
#8  0x000055555ec82eee in ?? ()
#9  0x000055555ec825f8 in ?? ()
#10 0x000055555ec78c99 in ?? ()
#11 0x0000555562d1a9be in ?? ()
#12 0x000055555ed6d573 in ?? ()
#13 0x0000555562c57ead in ?? ()
#14 0x0000555562c5944e in ?? ()
#15 0x000055555ed70729 in ?? ()
#16 0x000055555ee1c09e in ?? ()
#17 0x000055555ee1c332 in ?? ()
#18 0x000055555ed6e16a in ?? ()
#19 0x000055555a08212f in main ()
(gdb) 

Start Using gdb

Start gdb with mame, loading mamed directly will auto load the symbols

gdb ./mamed
--- snip ---
Reading symbols from ./mame...

Confirm mame is loaded with list

(gdb) list
1	// license:BSD-3-Clause
2	// copyright-holders:Aaron Giles
3	/***************************************************************************
4	
5	    mame.cpp
6	
7	    Specific (per target) constants
8	
9	****************************************************************************/
10	
(gdb) 

Run mame in windowed mode from within gdb (any mame params will work here so use as you like)

run -window

Perform the actions you need to make MAME fail for debugging and the gdb will halt execution

--Type <RET> for more, q to quit, c to continue without paging--

Press the Return to drop to a prompt

Thread 1 "mamed" received signal SIGBUS, Bus error.
0x00007ffff242f776 in ?? () from /lib/x86_64-linux-gnu/libGLX_mesa.so.0
(gdb) 


View a backtrace of the failure with the bt command

Thread 1 "mamed" received signal SIGBUS, Bus error.
0x00007ffff242f776 in ?? () from /lib/x86_64-linux-gnu/libGLX_mesa.so.0
(gdb) bt
#0  0x00007ffff242f776 in ?? () from /lib/x86_64-linux-gnu/libGLX_mesa.so.0
#1  0x00007ffff2444c96 in ?? () from /lib/x86_64-linux-gnu/libGLX_mesa.so.0
#2  0x00007ffff24331d9 in ?? () from /lib/x86_64-linux-gnu/libGLX_mesa.so.0
#3  0x00007ffff7f25331 in ?? () from /lib/x86_64-linux-gnu/libSDL2-2.0.so.0
#4  0x00007ffff7e9d9cf in ?? () from /lib/x86_64-linux-gnu/libSDL2-2.0.so.0
#5  0x000055555f6a732f in renderer_sdl1::~renderer_sdl1() ()
#6  0x000055555f6a741d in renderer_sdl1::~renderer_sdl1() ()
#7  0x000055555f670ea7 in sdl_window_info::~sdl_window_info() ()
#8  0x000055555f67a4a9 in sdl_event_manager::process_window_event(running_machine&, SDL_Event&) ()
#9  0x000055555f67a7ee in sdl_event_manager::process_events(running_machine&) ()
#10 0x000055555f679e88 in sdl_osd_interface::poll_inputs(running_machine&) ()
#11 0x000055555f66f91e in sdl_osd_interface::input_update() ()
#12 0x0000555563a0f507 in video_manager::frame_update(bool) ()
#13 0x000055555f769253 in mame_machine_manager::create_ui(running_machine&) ()
#14 0x000055556394374d in running_machine::start() ()
#15 0x0000555563944d00 in running_machine::run(bool) ()
#16 0x000055555f76c549 in mame_machine_manager::execute() ()
#17 0x000055555f81c2ff in cli_frontend::start_execution(mame_machine_manager*, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) ()
#18 0x000055555f81c592 in cli_frontend::execute(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&) ()
#19 0x000055555f769f2a in emulator_info::start_frontend(emu_options&, osd_interface&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&) ()
#20 0x000055555a08697f in main () at ../../../../../src/mame/mame.cpp:23
(gdb) 

You probably noted in the above output we are missing symbols for some systems libraries (sdl and mesa) to debug further we will need to enabled the appropriate symbols for these libraries.

Include the backtrace output in your report to assist with further troubleshooting

Final thoughts

Don't stress

The MAMEdev team know this source, you may not and it is a huge project to jump into. Raise an issue with your findings, a dev may make suggestions on where to look next or you may have managed to do 70% of the investigation for them, they do appreciate it.

Own your ignorance and ask for help, it is a learning opportunity that may have long term impact.