* GPU: Migrate buffers on GPU project, pre-emptively flush device local mappings
Essentially retreading #4540, but it's on the GPU project now instead of the backend. This allows us to have a lot more control + knowledge of where the buffer backing has been changed and allows us to pre-emptively flush pages to host memory for quicker readback. It will allow us to do other stuff in the future, but we'll get there when we get there.
Performance greatly improved in Hyrule Warriors: Age of Calamity. Performance notably improved in TOTK (average). Performance for BOTW restored to how it was before #4911, perhaps a bit better.
- Rewrites a bunch of buffer migration stuff. Might want to tighten up how dispose stuff works.
- Fixed an issue where the copy for texture pre-flush would happen _after_ the syncpoint.
TODO: remove a page from pre-flush if it isn't flushed after a certain number of copies.
* Add copy deactivation
* Fix dependent virtual buffers
* Remove logging
* Fix format issues (maybe)
* Vulkan: Remove backing swap
* Add explicit memory access types for most buffers
* Fix typo
* Add device local force expiry, change buffer inheritance behaviour
* General cleanup, OGL fix
* BufferPreFlush comments
* BufferBackingState comments
* Add an extra precaution to BufferMigration
This is very unlikely, but it's important to cover loose ends like this.
* Address some feedback
* Docs
* rebase
* add methods Ryyjinx.Common EmbeddedResources and SteamUtils
* GAL changes - change SetData() methods and ThreadedTexture commands to use IMemoryOwner<byte> instead of SpanOrArray<byte>
* Ryujinx.Graphics.Texture: change texture conversion methods to return IMemoryOwner<byte> and allocate from ByteMemoryPool
* Ryujinx.Graphics.OpenGL: update ITexture and Texture-like types with SetData() methods to take IMemoryOwner<byte> instead of SpanOrArray<byte>
* Ryujinx.Graphics.Vulkan: update ITexture and Texture-like types with SetData() methods to take IMemoryOwner<byte> instead of SpanOrArray<byte>
* Ryujinx.Graphics.Gpu: update ITexture and Texture-like types with SetData() methods to take IMemoryOwner<byte> instead of SpanOrArray<byte>
* Remove now-unused SpanOrArray<T>
* post-rebase cleanup
* PixelConverter: remove unsafe modifier on safe methods, and remove one unnecessary cast
* use ByteMemoryPool.Rent() in GetWritableRegion() impls
* fix formatting, rename `ReadRentedMemory()` to `ReadFileToRentedMemory()``
* Texture.ConvertToHostCompatibleFormat(): dispose of `result` in Astc decode branch
* Add support for large sampler arrays on Vulkan
* Shader cache version bump
* Format whitespace
* Move DescriptorSetManager to PipelineLayoutCacheEntry to allow different pool sizes per layout
* Handle array textures with different types on the same buffer
* Somewhat better caching system
* Avoid useless buffer data modification checks
* Move redundant bindings update checking to the backend
* Fix an issue where texture arrays would get the same bindings across stages on Vulkan
* Backport some fixes from part 2
* Fix typo
* PR feedback
* Format whitespace
* Add some missing XML docs
* Add host tracked memory manager mode
* Skipping flush is no longer needed
* Formatting + revert unrelated change
* LightningJit: Ensure that dest register is saved for load ops that do partial updates
* avoid allocations when doing address space lookup
Add missing improvement
* IsRmwMemory -> IsPartialRegisterUpdateMemory
* Ensure we iterate all private allocations in range
* PR feedback and potential fixes
* Simplified bridges a lot
* Skip calling SignalMappingChanged if Guest is true
* Late map bridge too
* Force address masking for prefetch instructions
* Reprotection for bridges
* Move partition list validation to separate debug method
* Move host tracked related classes to HostTracked folder
* New HostTracked namespace
* Move host tracked modes to the end of enum to avoid PPTC invalidation
---------
Co-authored-by: riperiperi <rhy3756547@hotmail.com>
* WIP: Separate guest/host tracking + unaligned protection
Allow memory manager to define support for single byte guest tracking
* Formatting
* Improve docs
* Properly handle cases where the address space bits are too low
* Address feedback
* Implement virtual buffer copies
* Introduce TranslateAndCreateMultiBuffersPhysicalOnly, use it for copy and clear
* Rename VirtualBufferCache to VirtualRangeCache
* Fix potential issue where virtual range could exist in the cache, without a physical buffer
* Fix bug that could cause copy with negative size on CopyToDependantVirtualBuffer
* Remove virtual copy back for SyncAction
* GetData XML docs
* Make field readonly
* Fix virtual buffer modification tracking
* Remove CopyFromDependantVirtualBuffers from ExternalFlush
* Move things around a little to avoid perf impact
- Inline null check for CopyFromDependantVirtualBuffers
- Remove extra method call for SynchronizeMemoryWithVirtualCopyBack, prefer calling CopyFromDependantVirtualBuffers separately
* Fix up XML doc
---------
Co-authored-by: riperiperi <rhy3756547@hotmail.com>
* WIP barrier batch
* Add store op to image usage barrier
* Dispose the barrier batch
* Fix encoding?
* Handle read and write on the load op barrier.
Load op consumes read accesses but does not add one, as the only other operation that can read is another load.
* Simplify null check
* Insert barriers on program change in case stale bindings are reintroduced
* Not sure how I messed this one up
* Improve location of bindings barrier update
This is also important for emergency deferred clear
* Update src/Ryujinx.Graphics.Vulkan/BarrierBatch.cs
Co-authored-by: Mary Guillemard <thog@protonmail.com>
---------
Co-authored-by: Mary Guillemard <thog@protonmail.com>
* Add a separate device memory manager
* Still need this
* Device writes are always tracked
* Device writes are always tracked (2)
* Rename more instances of gmm to mm
* Pass MultiRange to BufferManager
* Implement support for multi-range buffers using Vulkan sparse mappings
* Use multi-range for remaining buffers, delete old methods
* Assume that more buffers are contiguous
* Dispose multi-range buffers after they are removed from the list
* Properly init BufferBounds for constant and storage buffers
* Do not try reading zero bytes data from an unmapped address on the shader cache + PR feedback
* Fix misaligned sparse buffer offsets
* Null check can be simplified
* PR feedback
* Implement vertex and geometry shader conversion to compute
* Call InitializeReservedCounts for compute too
* PR feedback
* Set clip distance mask for geometry and tessellation shaders too
* Transform feedback emulation only for vertex
* Initial implementation of buffer mirrors
Generally slower right now, goal is to reduce render passes in games that do inline updates
Fix support buffer mirrors
Reintroduce vertex buffer mirror
Add storage buffer support
Optimisation part 1
More optimisation
Avoid useless data copies.
Remove unused cbIndex stuff
Properly set write flag for storage buffers.
Fix minor issues
Not sure why this was here.
Fix BufferRangeList
Fix some big issues
Align storage buffers rather than getting full buffer as a range
Improves mirrorability of read-only storage buffers
Increase staging buffer size, as it now contains mirrors
Fix some issues with buffers not updating
Fix buffer SetDataUnchecked offset for one of the paths when using mirrors
Fix buffer mirrors interaction with buffer textures
Fix mirror rebinding
Move GetBuffer calls on indirect draws before BeginRenderPass to avoid draws without render pass
Fix mirrors rebase
Fix rebase 2023
* Fix crash when using stale vertex buffer
Similar to `Get` with a size that's too large, just treat it as a clamp.
* Explicitly set support buffer as mirrorable
* Address feedback
* Remove unused fragment of MVK workaround
* Replace logging for staging buffer OOM
* Address format issues
* Address more format issues
* Mini cleanup
* Address more things
* Rename BufferRangeList
* Support bounding range for ClearMirrors and UploadPendingData
* Add maximum size for vertex buffer mirrors
* Enable index buffer mirrors
Enabled on all platforms for the IbStreamer.
* Feedback
* Remove mystery BufferCache change
Probably macos related?
* Fix mirrors not creating when staging buffer is empty.
* Change log level to debug
This branch changes the buffer copy fast path to notify memory tracking for all resources that aren't buffers. This fixes cases where games would copy buffer data directly into texture memory, which before would only work if the texture did not already exist. I imagine this happens when the guest driver is moving data between allocations or uploading it.
Since this only affects the fast path, cases where the source data has been modified from GPU (fast path copy destination doesn't count) will still fail to notify the texture, though I don't imagine games will do this. This should be resolved in future.
This should fix some texture issues with guest OpenGL games on switch, such as Dragon Quest Builders.
This may also be useful in future for games that move shader data around memory, if we end up using memory tracking for those.
* GPU: Don't sync/bind index buffer when it's not in use
Sometimes draws don't use an index buffer. It's not necessary to check or upload data for the current index buffer binding as it won't be used.
This fixes Pokemon: Legends Arceus updating a stale index buffer for every draw during its TFB pass, which was all non-indexed draws.
This probably didn't cost much on normal PCs, but it had a large impact on MacOS, which the macos1 release build avoided by mirroring index buffers (the PR currently does not). Needs buffer mirrors still for the rest of the performance.
There are additional cases where index buffers are bound or checked with non-indexed draws on the backend, but this one was straightforward to fix and has the largest impact. Testing is welcome to ensure nothing weird broke.
* Fix case with _rebind
* Fix incorrect fragment origin when YNegate is enabled
* Shader cache version bump
* Do not update support buffer if shader does not read gl_FragCoord
* Pass unscaled viewport size to the support buffer
* Add workflow to perform automated checks for PRs
* Downgrade Microsoft.CodeAnalysis to 4.4.0
This is a workaround to fix issues with dotnet-format.
See:
- https://github.com/dotnet/format/issues/1805
- https://github.com/dotnet/format/issues/1800
* Adjust editorconfig to be more compatible with Ryujinx code-style
* Adjust .editorconfig line endings to match .gitattributes
* Disable 'prefer switch expression' rule
* Remove naming styles
These are the default rules, so we don't need to override them.
* Silence IDE0060 in .editorconfig
* Slightly adjust .editorconfig
* Add lost workflow changes
* Move .editorconfig comment to the top
* .editorconfig: private static readonly fields should be _lowerCamelCase
* .editorconfig: Remove alignment for declarations as well
* editorconfig: Add rule for local constants
* Disable CA1822 for HLE services
* Disable CA1822 for ViewModels
Bindings won't work with static members, but this issue is silently ignored.
* Run dotnet format for the whole solution
* Check result code of SDL_GetDisplayBounds
* Fix dotnet format style issues
* Add missing trailing commas
* Update Microsoft.CodeAnalysis.CSharp to 4.6.0
Skipping 4.5.0 since it breaks dotnet format
* Restore old default naming rules for dotnet format
* Add naming rule exception for CPU tests
* checks: Include all files before excluding paths
* Fix dotnet format issues
* Check dotnet format version
* checks: Run dotnet format with severity info again
* checks: Disable naming style rules until they won't crash the process anymore
* Remove unread private member
* checks: Attempt to run analyzers 3 times before giving up
* checks: Enable naming style rules again with the new retry logic
* Move support buffer update out of the backends
* Fix render scale init and remove redundant state from SupportBufferUpdater
* Stop passing texture scale to the backends
* XML docs for SupportBufferUpdater
* dotnet format style --severity info
Some changes were manually reverted.
* dotnet format analyzers --serverity info
Some changes have been minimally adapted.
* Restore a few unused methods and variables
* Silence dotnet format IDE0060 warnings
* Silence dotnet format IDE0052 warnings
* Address dotnet format CA1816 warnings
* Address or silence dotnet format CA1069 warnings
* Address or silence dotnet format CA2211 warnings
* Address remaining dotnet format analyzer warnings
* Address review comments
* Address most dotnet format whitespace warnings
* Apply dotnet format whitespace formatting
A few of them have been manually reverted and the corresponding warning was silenced
* Format if-blocks correctly
* Run dotnet format whitespace after rebase
* Run dotnet format style after rebase
* Another rebase, another dotnet format run
* Run dotnet format style after rebase
* Run dotnet format after rebase and remove unused usings
- analyzers
- style
- whitespace
* Disable 'prefer switch expression' rule
* Add comments to disabled warnings
* Remove a few unused parameters
* Replace MmeShadowScratch with Array256<uint>
* Simplify properties and array initialization, Use const when possible, Remove trailing commas
* Start working on disabled warnings
* Fix and silence a few dotnet-format warnings again
* Run dotnet format after rebase
* Address IDE0251 warnings
* Silence IDE0060 in .editorconfig
* Revert "Simplify properties and array initialization, Use const when possible, Remove trailing commas"
This reverts commit 9462e4136c0a2100dc28b20cf9542e06790aa67e.
* dotnet format whitespace after rebase
* First pass of dotnet format
* Add unsafe dotnet format changes
* Fix typos
* Add trailing commas
* Disable formatting for FormatTable
* Address review feedback
* Implement transform feedback emulation for hardware without native support
* Stop doing some useless buffer updates and account for non-zero base instance
* Reduce redundant updates even more
* Update descriptor init logic to account for ResourceLayout
* Fix transform feedback and storage buffers not being updated in some cases
* Shader cache version bump
* PR feedback
* SetInstancedDrawVertexCount must be always called after UpdateState
* Minor typo
* Implement storage buffer operations using new Load/Store instruction
* Extend GenerateMultiTargetStorageOp to also match access with constant offset, and log and comments
* Remove now unused code
* Catch more complex cases of global memory usage
* Shader cache version bump
* Extend global access elimination to work with more shared memory cases
* Change alignment requirement from 16 bytes to 8 bytes, handle cases where we need more than 16 storage buffers
* Tweak preferencing to catch more cases
* Enable CB0 elimination even when host storage buffer alignment is > 16 (for Intel)
* Fix storage buffer bindings
* Simplify some code
* Shader cache version bump
* Fix typo
* Extend global memory elimination to handle shared memory with multiple possible offsets and local memory
* GPU: Avoid using garbage size for non-cb0 storage buffers
In the depths area, Tears of the Kingdom uses a global memory access with address on constant buffer slot 6. This isn't standard and thus doesn't actually have a size 8 bytes after it, so we were reading back a garbage size that ended up very large (at least in version 1.1.0), and would synchronize a lot of data per frame.
This PR makes storage buffers created from addresses outside constant buffer slot 0 get their size as the number of bytes remaining in the GPU mapping starting at the given virtual address. This should bound the buffer to a reasonable size, and ideally stop it crossing into other memory.
* Limit max size
* Add TODO
* Feedback
* GPU: Remove CPU region handle containers.
Another one for the "I don't know why I didn't do this earlier" pile.
This removes the "Cpu" prefixed region handle classes, which each mirror a region handle type from Ryujinx.Memory.
Originally, not all projects had a reference to Ryujinx.Memory, so these classes were introduced to bridge the gap. Someone else crossed that bridge since, so these classes don't have much of a purpose anymore.
This PR replaces all uses of CpuRegionHandle etc to their direct Ryujinx.Memory versions.
RegionHandle methods (specifically QueryModified) are about the hottest path there is in the entire emulator, so there is a nice boost from doing this.
* Add docs
* GPU: Allow granular buffer updates from the constant buffer updater
Sometimes, constant buffer updates can't be avoided, either due to a cb0 access that cannot be eliminated, or the game updating a buffer between draws to the detriment of everyone.
To avoid uploading the full 4096 bytes each time, this PR remembers the offset and size containing all constant buffer updates since the last sync. It will then upload that range after sync.
* Allow clearing the dirty range
* Always use precise
Might want to not do this if distance between the existing range and new one is too high.
* Use old force dirty mechanism when distance between regions is too great
* Update src/Ryujinx.Graphics.Gpu/Memory/Buffer.cs
Co-authored-by: gdkchan <gab.dark.100@gmail.com>
* Fix inheritance of _dirtyStart and _dirtyEnd
---------
Co-authored-by: gdkchan <gab.dark.100@gmail.com>
* WIP texture pre-flush
Improve performance of TextureView GetData to buffer
Fix copy/sync ordering
Fix minor bug
Make this actually work
WIP host mapping stuff
* Fix usage flags
* message
* Cleanup 1
* Fix rebase
* Fix
* Improve pre-flush rules
* Fix pre-flush
* A lot of cleanup
* Use the host memory bits
* Select the correct memory type
* Cleanup TextureGroupHandle
* Missing comment
* Remove debugging logs
* Revert BufferHandle _value access modifier
* One interrupt action at a time.
* Support D32S8 to D24S8 conversion, safeguards
* Interrupt cannot happen in sync handle's lock
Waitable needs to be checked twice now, but this should stop it from deadlocking.
* Remove unused using
* Address some feedback
* Address feedback
* Address more feedback
* Address more feedback
* Improve sync rules
Should allow for faster sync in some cases.
* GPU: Fix errors handling texture remapping
- Fixes an error where a pool entry and memory mapping changing at the same time could cause a texture to rebind its data from the wrong GPU VA (data swaps)
- Fixes an error where the texture pool could act on a mapping change before the mapping has actually been changed ("Unmapped" event happens before change, we need to signal it changed _after_ it completes)
TODO: remove textures from partially mapped list... if they aren't.
* Add Remap actions for handling post-mapping behaviours
* Remove unused code.
* Address feedback
* Nit
