Reorder our sync_fence_install() calls to happen after
all possible failures so that error cleanup will be
correct.
CRs-Fixed: 1081855
Change-Id: I0e7bb459f2acc010446ac5e5b3b72c8b16cce079
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Signed-off-by: Sudeep Yedalapure <sudeepy@codeaurora.org>
There is a possible race condition where the process can be going
away while its debugfs 'mem' file is being read, which could cause
memory corruption.
Conflicts:
drivers/gpu/msm/kgsl.c
CRs-Fixed: 627780
Change-Id: I697486faeb3f186fd1220d0acc1e449a4f7b77b0
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Signed-off-by: Hareesh Gundu <hareeshg@codeaurora.org>
For contexts created with the KGSL_CONTEXT_USER_GENERATED_TS,
allow events to be created for timestamps that have not
been issued yet. Presumably these contexts know what they
are doing.
Change-Id: Iccf2e549b38e1a11850d26ee64b7279ab0b8a1ae
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
On targets where trustzone is available, use the trustzone
based governor instead of simple_ondemand.
Change-Id: Ie11c57684fc63a26e82a5c4816a35a327a1bf945
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Signed-off-by: Vladimir Razgulin <vrazguli@codeaurora.org>
Convert the clock frequency scaling infrastructure to
be based on devfreq.
Change-Id: I1a60ba339db5715a8836b835bd1b29b46e151af6
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Signed-off-by: Vladimir Razgulin <vrazguli@codeaurora.org>
This governor can be used to control adreno GPUs.
It is unlikely to be useful for other devices.
Change-Id: Icf481322454b814d2f41019f2f01286062409952
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Signed-off-by: Vladimir Razgulin <vrazguli@codeaurora.org>
Adreno context destruction never needs the hardware on
to destroy the current context.
For z180, always take an active count while destroying.
Since this is the last ioctl to use KGSL_IOCTL_WAKE, remove
it entirely.
Change-Id: I97368c7f3c28f16538cc5f48b802b681d7af96ef
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Most of the adreno specific ioctls do not need the hardware
on. The few that do are perfcounter related, it may be
better at some point to optimize them further to shadow
the counters when the hardware is not on rather than
forcing the power on.
Change-Id: I9ea83106e0aaf3d3ac8d261bdec4e4c2e21c4659
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Getting an active count can fail if the hardware doesn't
actually wake up. Make active count get functions __mustcheck
to remind people of this and fix existing calls to
handle the error.
Change-Id: I5a0202c824ac7e436b9061da6d8f638e44e8b7f6
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Store flags from userspace context creation in kgsl_context.flags,
and move all internal flags to adreno_context.priv.
Change-Id: I3c73ebee4092abf5864238319c214c4d977bdaad
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Move logic for handling preamble based context switch
to the core adreno code. This makes it less burdensome
to implement support for newer GPU families that won't
ever support legacy context switching.
Change-Id: Id9ad5936ff91dcdbc9de869baf0d0b9fcf1b5170
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Checking NULL before derefencing usually works better.
Change-Id: I2e2936eb9532dae4b0ef471b5cb2dd1c3a8f0975
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
It isn't possible to use rcu_read_lock() sections to guard
access to a data structure that is refcounted with a kref.
Rather than creating RCU-aware refcounts for kgsl_mem_entry
as described in Documentation/RCU/rcuref.txt, just use
the mem_lock to guard lookups in the idr.
Change-Id: Ia0733b156fc7a9b446cb8221b9172ce9faf111e7
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Having a separate allocated struct for the device specific context
makes ownership unclear, which could lead to reference counting
problems or invalid pointers. Also, duplicate members were
starting to appear in adreno_context because there wasn't a safe
way to reach the kgsl_context from some parts of the adreno code.
This can now be done via container_of().
This change alters the lifecycle of the context->id, which is
now freed when the context reference count hits zero rather
than in kgsl_context_detach().
It also changes the context creation and destruction sequence.
The device specific code must allocate a structure containing
a struct kgsl_context and passes a pointer it to kgsl_init_context()
before doing any device specific initialization. There is also a
separate drawctxt_detach() callback for doing device specific
cleanup. This is separate from freeing memory, which is done
by the drawctxt_destroy() callback.
Change-Id: I7d238476a3bfec98fd8dbc28971cf3187a81dac2
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
We really don't want new GPU commands or events to be generated
just to manage the iommu while we are idle.
Change-Id: I2e8740bee8c25c93bddc51a90a3370d151aaf558
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
These two functions need to agree on the meaning of "idle",
so that calling adreno_isidle() right after an adreno_idle()
will always return true.
Change-Id: I7cddf73773186c3ec8b56c111affacac3b07fcc7
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Make full_cache_threshold==0 mean never do a full cache flush,
since that is more useful than having it mean always do it.
Change-Id: I4ae46b6253f2d78a97f067b5f906637a3db1d5c8
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Use the same format for all trace points reporting the same
data, such as context ids (ctx=%u) and timestamps (ts=%u).
Make sure key=value format is used in a few places where it
was missing.
Change-Id: I4ec1c77c853c567c7a6ba69eff5023d8d71cdac4
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Since the rptr is written by the GPU, there's no point
in keeping a copy in the ringbuffer struct where it will
likely be out of date. If you need to look at the ringbuffer,
read it into a local variable with adreno_get_rptr().
Change-Id: Ibf1ba0b9c71a93f65a5c85a58328b2202a27af3f
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Level based governors may need to perform this lookup to
interpret the current frequency of the device.
Change-Id: Idf7246b05775a52f088c52b898d98fbab4fd942c
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Signed-off-by: Vladimir Razgulin <vrazguli@codeaurora.org>
This field is treated as governer specific data for
a devfreq instance. But there's currently no way to
set the correct data when switching governors through
sysfs. Add support for optionally passing a set of
name / data pairs in struct devfreq_dev_profile,
representing the data for each governor.
Change-Id: I5523ce94f8b0045974f0635fb734cb1282512f91
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Signed-off-by: Vladimir Razgulin <vrazguli@codeaurora.org>
On 8064 and 8974, flushing more than 16mb of virtual address
space is slower than flushing the entire cache. So flush
the entire cache when the working set is larger than this.
The threshold for full cache flush can be tuned at runtime via
the full_cache_threshold sysfs file.
Change-Id: If525e4c44eb043d0afc3fe42d7ef2c7de0ba2106
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Add a new ioctl, IOCTL_KGSL_GPUMEM_SYNC_CACHE_BULK, which can be used
to sync a number of memory ids at once. This gives the driver an
opportunity to optimize the cache operations based on the total
working set of memory that needs to be managed.
Change-Id: I9693c54cb6f12468b7d9abb0afaef348e631a114
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
The device state can fluctuate during recovery and resume, so
it is safer to check the gates unconditionally rather than
trust the state. The gates are left open until recovery
or suspend starts.
Change-Id: Idad91976ceba0904425e54c71faf88b4fccb8fdd
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
When KGSL_MEMFLAGS_USE_CPU_MAP is set, we must check that the
address from get_unmapped_area() is not used as part of a
mapping that is present only in the GPU pagetable and not the
CPU pagetable. These mappings can occur because when a buffer
is freed on timestamp, the CPU mapping is destroyed immediately
but the GPU mapping is not destroyed until the GPU timestamp
has passed.
Because kgsl_mem_entry_detach_process() removed the rbtree
entry before removing the iommu mapping, there was a window
of time where kgsl thought the address was available even
though it was still present in the iommu pagetable. This
could cause the address to get assigned to a new buffer,
which would cause iommu_map_range() to fail since the old
mapping was still in the pagetable. Prevent this race by
removing the iommu mapping before removing the rbtree entry
tracking the address.
Change-Id: I8f42d6d97833293b55fcbc272d180564862cef8a
CRs-Fixed: 480222
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Imported memory buffers sometimes do not have enough
padding to prevent page faults due to overzealous
GPU prefetch. Attach guard pages to their mappings
to prevent these faults.
Because we don't create the scatterlist for some
types of imported memory, such as ion, the guard
page is no longer included as the last entry in
the scatterlist. Instead, it is handled by
size ajustments and a separate iommu_map() call
in the kgsl_mmu_map() and kgsl_mmu_unmap() paths.
Change-Id: I3af3c29c3983f8cacdc366a2423f90c8ecdc3059
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Make sure memory does not get freed twice if one of the
frees is free on timestamp.
Change-Id: Id03d69ae9b5cc598658794e65152a899b36fe314
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Make kgsl_sharedmem_find* return a reference to the
entry that was found. This makes using an entry
without the mem_lock held less race prone.
Change-Id: If6eb6470ecfea1332d3130d877922c70ca037467
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Remove duplicate log messages that were happening for
every iommu page fault.
Change-Id: I13bebcd3e93165d1b22ca859c39d0b169d3a53eb
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
If msm_iommu_map_range() fails mid way through the va
range with an error, clean up the PTEs that have already
been created so they are not leaked.
Change-Id: Ie929343cd6e36cade7b2cc9b4b4408c3453e6b5f
CRs-Fixed: 478304
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
When KGSL_MEMFLAGS_USE_CPU_MAP is enabled, the mmap address
must try to match the GPU alignment requirements of the buffer,
as well as include space in the mapping for the guard page.
This can cause -ENOMEM to be returned from get_unmapped_area()
when there are a large number of mappings. When this happens,
fall back to page alignment and retry to avoid failure.
Change-Id: I2176fe57afc96d8cf1fe1c694836305ddc3c3420
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Require any code path which intends to touch the hardware
to take a reference on active_cnt with kgsl_active_count_get()
and release it with kgsl_active_count_put() when finished.
These functions now do the wake / sleep steps that were
previously handled by kgsl_check_suspended() and
kgsl_check_idle().
Additionally, kgsl_pre_hwaccess() will no longer turn on
the clocks, it just enforces via BUG_ON that the clocks
are enabled before a register is touched.
Change-Id: I31b0d067e6d600f0228450dbd73f69caa919ce13
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
These 2 counter groups are also "special cases" that require
different programming sequences.
Change-Id: I73e3e76b340e6c5867c0909b3e0edc78aa62b9ee
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Use the default setstate function, which directly reprograms the IOMMU,
to change IOMMU pagetables or flush the TLB if the GPU is already idle.
This condition often occurs when the GPU is being powered down. In this
case it is desirable to avoid the overhead of issuing commands, waiting
for idle and firing events that results from using the GPU
command stream to reprogram the IOMMU.
Change-Id: I633002ac49c8fe58df3f1f6a1fd1ddf705fc1733
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Make sure iommu_map_range() does not leave a partial
mapping on error if part of the range is already mapped.
Change-Id: I108b45ce8935b73ecb65f375930fe5e00b8d91eb
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
This allows address space and pagetable configuration to be
set from data in the mmu, such as devtree settings.
Change-Id: I811b2d8bbac2613a0ea51795f7cd4b121ec203da
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Make sure cache operations don't hit pagefaults by
backing the entire vma in mmap() instead of faulting
in pages as they're touched. Otherwise, there's a
chance that a later cache operation on the mapping
could trigger an unhandled page fault leading to
a kernel panic.
Change-Id: Ia73c8aaed2708c5b9ef46ed50fb0f5cf1ad2450c
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
The guard page needs to be readable by the GPU, due to
a prefetch range issue, but it should never be writable.
Change the page fault message to indicate if nearby
buffers have a guard page.
Change-Id: I3955de1409cbf4ccdde92def894945267efa044d
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
The two flags fields in kgsl_memdesc should be enough for
anyone. Move the only flag using kgsl_mem_entry, the
FROZEN flag for snapshot procesing, to use kgsl_memdesc.priv.
Change-Id: Ia12b9a6e6c1f5b5e57fa461b04ecc3d1705f2eaf
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Make kgsl_memdesc_protflags() return the correct type of flags
for the type of mmu being used. Query the memdesc with this
function in kgsl_mmu_map(), rather than passing in the
protflags. This prevents translation at multiple layers of
the code and makes it easier to enforce that the mapping matches
the allocation flags.
Change-Id: I2a2f4a43026ae903dd134be00e646d258a83f79f
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
The iommu hardware versioning scheme recently changed in the iommu driver.
Rename version specific defines and structures in kgsl to match.
Change-Id: Ib2ca5e40d8f5ed6c79b9f4f5a013263fd8d4aae4
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Add the event kgsl_mem_sync_cache. This event is
emitted when only a cache operation is actually
performed. Attempts to flush uncached memory,
which do nothing, do not cause this event.
Change-Id: Id4a940a6b50e08b54fbef0025c4b8aaa71641462
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Limit the amount of pages vmapped at one time during allocation,
to avoid requesting more vmalloc space than is likely available.
CRs-Fixed: 445005
Change-Id: Icee5912687edf4da585d88308ee0e1c971964785
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
KGSL_GPUMEM_ALLOC_ID now takes a flag,
KGSL_MEMFLAGS_USE_CPU_MAP. When set, the GPU
mapping will be set up to match the CPU mapping
during mmap(). This feature is only supported when
using per process pagetables with the IOMMU. The
flags field of KGSL_GPUMEM_ALLOC_ID is copied back
to userspace and KGSL_MEMFLAGS_USE_CPU_MAP will
be cleared when this feature is not supported.
The IOMMU virtual address space has been adjusted
when perprocess pagetables is enabled so that the
entire userpace address range (0 to TASK_SIZE) can
have equivalent mappings on the IOMMU. For buffers
that do not have equivalent mappings, the address
range from PAGE_OFFSET to KGSL_IOMMU_GLOBAL_MEM_BASE
is used.
Change-Id: Ib61c03aa7453c3dd901c41e8fd297f66d402ae1a
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Previously, the gpu address has been used to uniquely
identify each memory allocation. Upcoming patches will
introduce cases where an allocation does not always
have a gpu address, so an additional id is needed.
IOCTL_KGSL_GPUMEM_ALLOC_ID allocates pages and returns
an id.
IOCTL_KGSL_GPUMEM_FREE_ID frees an id. KGSL_SHAREDMEM_FREE
can still be used to free by GPU address, if it exists.
The id can also be passed to mmap(), shifted left by
PAGE_SIZE to get a CPU mapping for the buffer.
IOCTL_KGSL_GPUMEM_ALLOC_GET_INFO can be called to retrieve
the id and other information about the buffer.
Change-Id: I4b45f0660cb9d4a5fb1323ccc6c4aa360791c1ec
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
The v1 iommu only supports splitting between TTBR0 and TTBR1
on a power of two boundary. Cutting off the userspace address
at 2G (0x80000000) is inconvienient, as the GPU userspace
address space should align with the CPU address space.
This requires changing how global allocations are managed,
since there is no longer a separate pagetable for TTBR1.
The default pagetable is still the master of these allocations
and maintains the gen_pool for allocating global addresses.
But now, these regions are mapped into each process pagetable
by calling kgsl_setup_pt(). This requires kgsl_mmu_map
and kgsl_mmu_unmap to be able to handle mapping without
virtual address allocation.
Change-Id: I94e2d63dc7e6a7ef576f993770725b6b7ba14228
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Make sure iommu_map_range() does not leave a partial
mapping on error if part of the range is already
mapped.
Change-Id: I0ddeb0e0169b579f1efdeca4071fce4ee75a11f8
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
msm_soc_version_supports_iommu_v1() was defined outside the guard.
Change-Id: I8db106908b08b89e267550d81d031cdb028b92a2
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
These cases have never been a normal operation pattern of the
userspace driver. Sharing buffers through multiple mappings
is better left to dma-buf or ion.
Change-Id: I7e7658137937c96b9505d0f912dcb262d652e0c3
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Some external memory types were using hostptr to store
a userspace virtual address, but other code assumes it is a
kernel virtual address. Make memdesc->hostpr always
be the kernel virtual address and add a useraddr
field for the userspace virtual address.
Change-Id: Id4580a2ff34aeb15f2c1b26a7134f0fd4ec52a6e
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Userspace passes a flag KGSL_MEMFLAGS_GPUREADONLY when
the gpu should not write to a memory region. Use this
flag to control IOMMU_WRITE permissions on the buffer.
Change-Id: I5d3fc615dc36687252e2242f63fe74d6ce1c4fbc
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Fix a possible crash in adreno_waittimestamp() when
waiting on the global timestamp.
Change-Id: I7d50d3298962c4d8691dc0807438d5ab86cbc477
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Add a file for each process in debugfs named, kgsl/proc/<pid>/mem
which contains information about all memory allocations the process
has made.
Change-Id: Ice3f039d92cc1b1cdb5a6192808441ddfdf8abfb
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Add a "usage" field to memory allocation, mapping and
free ftrace events.
Change-Id: I673a9593650d5285b0abc8c94de8f9f80d3d449e
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Userspace passes a set of values indicating how it
uses each buffer it allocates, which were previously
ignored. These are useful hints for debugging and
profiling applications. These flags will be exposed
through ftrace and debugfs in later patches.
Change-Id: Ie26c26e413c074dcd5dfa24d355443ee47c3cd6a
Signed-off-by: Shubhraprakash Das <sadas@codeaurora.org>
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Add kgsl_context_create and kgsl_context_detach
trace events for tracking context lifetime.
Include more timestamp information in existing
events.
Change-Id: I13afb1e816caa7b668cde31cdcc8a8bd1bf8b4dc
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Calling this device function directly from the function
table is unnecessarily verbose, and done fairly often.
Change-Id: Ib9d75b31dfab8fb4ccced46fe62a08a98da8c94f
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
It is often very useful to see where a pagefault
occurs in relation to other trace events such
as kgsl_mem_free or kgsl_issueibcmds, so that the
source of the pagefault can be isolated.
Change-Id: I846cb588e1b09bf12698d014d289b64f35576c7e
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Cancel per context events after the device specific
portion of the context is destroyed and cancel
global timestamp events after all the contexts
are destroyed. This reduces the likelyhood that memory
will be freed while the gpu is still using it.
CRs-Fixed: 361864
Change-Id: If798c5dd0418167e2a091bb58d810dcbd0d59154
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Signed-off-by: Ranjhith Kalisamy <ranjhith@codeaurora.org>
Per-context timestamps introduced a race condition
between the context destroy ioctl and the waittimestamp
ioctl. The waittimestamp ioctl release device->mutex
while it is waiting to prevent deadlock. It also has
a context pointer, which could be freed from a different
thread while the waiting thread was blocked.
Fix this by adding a reference count to the context
structure, which must be held by any code that maintains
a pointer to a context while the device mutex is not
held. Currently this only happens via waittimestamp.
The "main" reference count removed by userspace
requesting the context to be destroyed. When this
happens kgsl_context_detach() is called, which does
a partial cleanup of the context so that it can
no longer be used to issue commands. Once a context
has been detached, its id field is set to
KGSL_CONTEXT_INVALID. Unfortunately this is needed
by adreno_waittimestamp() so it can correctly stop
waiting in this case. Cleaning up adreno_waittimestamp()
will need to be handled in a separate patch.
CRs-Fixed: 355155
Change-Id: Ib934b467cd077b5ee774de5f297660e418d693e5
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Clean up some redundant code and prevent gen_pool_free
from being called on the wrong pool for some error
cases in kgsl_mmu_map.
Change-Id: I0d783fbdf393de5d5481bd0270a042671b7d4e6b
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
All allocations made from the pool use the same alignment,
so it is simpler to just increase the overall pool order
to get the needed alignment.
Change-Id: I6de5069388bf8344d60902dedea795f7c5629f54
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Initialization code was split between kgsl_device_probe
and kgsl_register_device, which made the code unclear and
error handling difficult. Now all initialization of data
structures happens in probe, and kgsl_register_device only
manages the /dev file.
Change-Id: I85d49c305310b943ab7600762f1f230804a3b9a8
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
There are many fields in kgsl_device that are always initialized
the same way. Move them to a macro, KGSL_DEVICE_COMMON_INIT,
so the correct static initializers can be used by both 3d and 2d.
Change-Id: I8362957548199112517457e0f94bfe2bd5f707f9
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Add a225 specific registers to both snapshot and postmortem
output.
Change-Id: Ia0648af6296665ceb1c3354485a02a8950c92c87
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
It is much more useful to get the dword offset of the
failing register than the byte offset because it is
consistent with the #defines and other register
logging in postmortem dump and snapshot.
Change-Id: Ie257475a8e2224729e4d1ac14af13a8027974027
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Read and log the gpu virtual and physical address that MH thinks
caused the AXI error from the debug bus.
Change-Id: I2c381845f3e1e4f82bc42732c46a8ffa14658a92
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
ion handles need to be handled differently than pmem or
ashmem handles.
CRs-Fixed: 359268
Change-Id: I20637644f247859245fbba80e5bc6b50fd896403
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
/sys/class/kgsl/kgsl-3d0/snapshot/timestamp will have sysfs_notify()
called when a hang happens, so that a userspace process can
poll(2) on this file to detect a hang.
CRs-Fixed: 354385
Change-Id: I0a3c8fcbe3fb09256bcd12f6e63c107536734485
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
The 2d timestamp is used as an index into the ringbuffer.
Now that z180_dev->current_timestamp is preserved across
resume, make sure that ADDR_VGV3_NEXTADDR is set to the
gpu address that corresponds to the current timestamp.
CRs-Fixed: 347025
Change-Id: Ib7dc4579943ab24e04279ede1555539ec21adabd
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
The outer cache needs to be flushed for these pages
after they are allocated so that the GPU and CPU
have a consistent view of them.
Change-Id: I4d6b688fc4a9f04d4bf8e3215d51bac32bf8e9fd
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
The 2d hardware handles ringbuffer and IB commands as
a series of gotos. At the end of each IB, there must
be a goto command back to the ringbuffer, which must
be "monkey patched" into the IB by the driver.
Fix this code to use a proper kernel mapping.
Change-Id: Ic35e6fbf6baeef51dbc2497f1702c7ccd6997579
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Separate ib parse checking from cffdump as it is useful
in other situations. This is controlled by a new debugfs
file, ib_check. All ib checking is off (0) by default,
because parsing and mem_entry lookup can have a performance
impact on some benchmarks. Level 1 checking verifies the
IB1's. Level 2 checking also verifies the IB2.
Change-Id: Ibf3c6d1e0d7522e75b41e1a6dbb92020ae9ace8d
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Ion carveout and content protect heap buffers do not
have a struct page associated with them. Thus
sg_phys() will not work reliably on these buffers,
so set dma_address on their scatterlists.
CRs-Fixed: 345257
Change-Id: Ifdad5ce497de170f47b4ee2f7a93563a5cbe1a96
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Ion carveout and content protect heap buffers do not
have a struct page associated with them. Thus
sg_phys() will not work reliably on these buffers.
Set the dma_address field on physically contiguous
buffers. When mapping a scatterlist to the gpummu
use sg_dma_address() first and if it returns 0
then use sg_phys().
Change-Id: Ie5f19986446be4383dfbfffa2534136b592e8e46
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Ion carveout and content protect buffers do not have
a struct page and thus sg_phys() cannot be used on them.
Try sg_dma_address() first and if it returns 0 then
use sg_phys().
Change-Id: I95ccb8f5a3c86cd09ecf2a2737c260f4996059ac
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Memory mapped through kgsl_mmu_map_global() is supposed to have
the same gpu address in all pagetables. And the memdesc will
persist beyond the lifetime of any single pagetable.
Therefore, memdesc->gpuaddr should not be zeroed for these
memdescs.
Change-Id: I0f46aaee2b9e87f839e78b7978cdf1bb4239d6f5
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Postmortem dump was not parsing CP_INDIRECT_BUFFER_PFE commands.
Snapshot was recently fixed to handle this, and this change
extends support to postmortem dump.
Change-Id: I07775ef4449efabc8cdebb1635835e7526b1c36e
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
This function is supposed to return the memdesc that
contains the range gpuaddr to gpuaddr + size. One of the
lookups was using sizeof(unsigned int) instead of size,
which could cause false positive results from this function
and possibly kernel panics in the snapshot or postmortem
code, which rely on it to do bounds checking for them.
Change-Id: I65dc48108f2010887e620a252a6afbd88473ac6e
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Add events for tracking memory operations by userspace
clients: kgsl_mem_alloc, kgsl_mem_map, kgsl_mem_free,
kgsl_mem_timestamp_queue (adding an entry to the free
on timestamp list) and kgsl_mem_timestamp_free (when
the memory is actually freed).
Change-Id: Id62eec30ea20a0f00f7a7a791c7e5b8dfad487af
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Having the snapshot buffer physically contiguous will make
it easier to recover from a ram dump in case the system
crashes after a hang. Also log the buffer address when
the snapshot is created so we know where to look for it.
Change-Id: I13fe603d0e9cb1118d15926ff5f8855420365c42
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Unknown ioctl code errors are supposed to be ENOIOCTLCMD,
not EINVAL.
Change-Id: Id38529e4ec70d63091a9273a780585aea6ae9d9a
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
The sysfs code was reading the wrong entry in the stats
array, so it printed the wrong value.
Change-Id: I26ce8bbca152a98e3dac53a9c154e52b020d5532
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
The timestamp memqueue was unsorted, which could cause
memory to not be freed soon enough. The kgsl_event
list is sorted and does almost exactly the same thing
as the memqueue did, so freememontimestamp is now
implemented using the kgsl_event list.
CRs-Fixed: 327647
Change-Id: Ia93431bc5f31a9286cc78a92271b43a3a6ac9baf
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Events need to be cancelled when an fd is released,
to avoid possible memory leaks or use after free.
When the event is cancelled, its callback is called.
Currently this is sufficient since events are used for
resource management and we have no option but to
release the lock or memory. If future uses need to
distinguish between the callback firing and
a cancel, they can look at the timestamp passed to
the callback, which will be before the timestamp they
expected. Otherwise a separate cancel callback can
be added.
CRs-Fixed: 327647
Change-Id: I49f8f58aea6366344d1c82613f73881a169834b3
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
For dma_alloc_coherent() you don't need writel/readl because
it's just a plain old void *. Linux tries very hard to make a
distinction between io memory (void __iomem *) and memory
(void *) so that drivers are portable to architectures that
don't have a way to access registers via pointer dereferences.
You can see http://lwn.net/Articles/102232/ and the Linus rant
http://lwn.net/Articles/102240/ here for more details behind
the motivation.
Change-Id: I3da075c30304e4adf321cfb3edb1baa4a93fc2ce
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
There are a some workloads where interrupts do not
always get generated, and as a result the timestamp
work was not triggered often enough.
Queue timestamp expired work from adreno_waittimestamp(),
when the timestamp expires while we are not waiting.
It is possible in this case that no interrupt fired
because no processes were waiting.
Queue timestamp expired work when freememontimestamp
is called, which reduces the amount of memory
built up by applications that use this api often.
CRs-Fixed: 336057
Change-Id: Ib4610dfa53af04b5a7b1eb51281d32d0e5be8471
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
These idles were necessary when the context switch code
preserved PM_OVERRIDE registers. That was removed some time
ago so now the idles are not needed.
CRs-Fixed: 331325
Change-Id: I90d4e1685f213a5720bf6da4e6d4a187d1eb0689
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Increase the initial pagetables created to 24 for 8660
and 8960. Testing shows that there can be about 20
processes using graphics at a time. When this happens
memory utilization will be great enough that an
attempt to allocate more coherent memory will fail.
This will cost an additional 2MB of coherent memory,
bringing the total usage for KGSL to 6MB.
CRs-Fixed: 331629
Change-Id: Ib9f7fe9c0cc0194ebbf5dfc69196e7cad7a44b11
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
The RBBM_STATUS loop is extremely tight and calling
adreno_poke() each time through this loop appears
to cause watchdog bites on 8960.
CRs-Fixed: 324507
Change-Id: I65145ca96f63280179d76abd5320fd201beeeaa7
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
If another state has been requested, there is no reason
to queue work.
Change-Id: Ifa8d57e523a047bd2021c00732c2fe6a67bd6790
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
These functions write registers and so if they are called
from kgsl_pwrctrl_irq() there is a chance that they
might be called when clocks are disabled.
Change-Id: I259d840d9996d424aba3e3481201d49f602637e8
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Clean up kgsl_pwrctrl_sleep() and kgsl_pwrctrl_wake() to make
state transistions clearer. Add kgsl_pwrctrl_request_state()
and kgsl_pwrctrl_set_state() to make it easier to debug the
state machine.
CRs-Fixed: 315833
Change-Id: I656ce8bd19feabd4186ef91dc031f8a6c6a7d09a
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
If the ringbuffer isn't idle, the GPU is probably in
SLUMBER or SUSPEND which are very idle states. This
function probably still shouldn't be called, so WARN
instead.
Change-Id: Ia49a819d935d6dce9cae2c193aff8e9de87d8af6
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
If postmortem dump tried to parse a context switch
buffer for any context except the current context
this function would never return.
Change-Id: I19f2fe7afda5796180cdf5f81c17fad960e84f05
Signed-off-by: Jeremy Gebben <jgebben@codeaurora.org>
Jeremy Gebben